Jul 10 12:43:53.697971 ip-10-0-143-162 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 12:43:53.697982 ip-10-0-143-162 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 12:43:53.697992 ip-10-0-143-162 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 12:43:53.698347 ip-10-0-143-162 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 12:44:03.166028 ip-10-0-143-162 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 10 12:44:03.166039 ip-10-0-143-162 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9c61d617e80d433e883ec60759288b96 -- Jul 10 12:45:49.009601 ip-10-0-143-162 systemd[1]: Starting Kubernetes Kubelet... Jul 10 12:45:49.361200 ip-10-0-143-162 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 12:45:49.361200 ip-10-0-143-162 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 12:45:49.361200 ip-10-0-143-162 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 12:45:49.361200 ip-10-0-143-162 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 12:45:49.361200 ip-10-0-143-162 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 12:45:49.362042 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.361966 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 12:45:49.366666 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366652 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 12:45:49.366666 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366667 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366670 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366673 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366675 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366678 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366681 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366686 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366689 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366691 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366693 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366696 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366698 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366701 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366703 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366705 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366708 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366710 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366713 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366715 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366717 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 12:45:49.366729 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366720 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366722 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366724 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366727 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366729 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366732 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366734 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366737 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366739 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366742 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366744 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366746 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366749 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366751 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366754 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366756 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366758 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366762 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366764 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366767 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 12:45:49.367196 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366769 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366771 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366773 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366776 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366778 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366780 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366782 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366785 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366787 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366789 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366791 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366793 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366796 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366798 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366801 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366804 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366806 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366809 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366811 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 12:45:49.367680 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366813 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366816 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366818 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366821 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366823 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366825 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366827 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366830 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366832 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366834 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366837 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366839 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366842 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366844 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366847 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366849 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366851 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366853 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366855 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366858 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 12:45:49.368094 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366862 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366865 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366868 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366870 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366874 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366876 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366879 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366882 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366884 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366887 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366889 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366892 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366894 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366898 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366901 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366905 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366908 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366910 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.366913 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 12:45:49.368554 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367619 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367630 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367633 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367637 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367639 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367643 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367645 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367648 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367650 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367653 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367656 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367658 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367660 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367663 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367665 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367667 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367670 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367672 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367675 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367677 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 12:45:49.369002 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367679 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367682 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367685 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367687 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367689 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367693 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367696 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367699 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367701 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367704 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367706 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367709 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367711 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367715 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367717 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367719 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367722 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367724 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367727 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367729 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 12:45:49.369498 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367731 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367733 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367736 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367738 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367740 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367743 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367745 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367748 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367750 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367752 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367754 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367757 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367759 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367762 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367764 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367766 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367769 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367771 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367774 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 12:45:49.369932 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367776 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367778 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367781 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367783 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367785 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367788 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367791 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367794 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367797 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367799 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367801 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367804 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367806 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367808 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367811 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367813 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367815 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367818 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367820 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 12:45:49.370399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367824 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367828 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367830 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367833 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367836 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367839 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367841 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367843 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367848 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367851 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367854 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367856 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367859 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367862 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367864 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367867 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367869 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367871 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367874 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 12:45:49.370828 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367878 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.367880 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368196 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368201 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368203 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368214 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368217 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368220 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368222 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368224 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368227 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368229 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368231 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368234 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368236 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368242 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368244 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368247 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368249 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368252 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 12:45:49.371250 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368254 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368256 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368259 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368261 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368264 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368266 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368268 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368271 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368273 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368276 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368278 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368285 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368298 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368301 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368306 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368310 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368312 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368315 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368317 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 12:45:49.371716 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368319 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368322 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368325 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368327 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368330 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368332 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368334 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368337 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368341 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368344 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368346 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368349 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368351 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368353 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368355 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368358 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368360 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368363 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368366 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368368 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 12:45:49.372125 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368370 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368373 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368375 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368377 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368380 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368382 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368384 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368387 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368389 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368392 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368395 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368399 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368401 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368404 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368407 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368410 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368413 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368415 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368418 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 12:45:49.372593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368421 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368425 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368427 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368429 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368432 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368434 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368436 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368439 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368441 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368444 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368446 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368448 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368451 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368453 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368456 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368458 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368460 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368463 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368465 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368467 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 12:45:49.373003 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368470 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368472 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.368475 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368531 2571 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368543 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368548 2571 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368552 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368556 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368560 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368564 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368568 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368570 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368573 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368577 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368581 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368584 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368587 2571 flags.go:64] FLAG: --cgroup-root="" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368589 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368592 2571 flags.go:64] FLAG: --client-ca-file="" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368594 2571 flags.go:64] FLAG: --cloud-provider="external" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368597 2571 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368601 2571 flags.go:64] FLAG: --cluster-domain="" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368604 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368607 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 12:45:49.373455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368610 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368613 2571 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368617 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368619 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368622 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368625 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368628 2571 flags.go:64] FLAG: --contention-profiling="false" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368631 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368634 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368637 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368639 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368644 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368646 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368649 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368652 2571 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368654 2571 flags.go:64] FLAG: --enable-server="true" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368657 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368661 2571 flags.go:64] FLAG: --event-burst="100" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368664 2571 flags.go:64] FLAG: --event-qps="50" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368667 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368669 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368673 2571 flags.go:64] FLAG: --eviction-hard="" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368677 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368679 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368682 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 12:45:49.373977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368685 2571 flags.go:64] FLAG: --eviction-soft="" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368687 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368690 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368693 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368695 2571 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368698 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368700 2571 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368703 2571 flags.go:64] FLAG: --feature-gates="" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368707 2571 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368710 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368712 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368715 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368718 2571 flags.go:64] FLAG: --healthz-port="10248" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368721 2571 flags.go:64] FLAG: --help="false" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368724 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368727 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368729 2571 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368732 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368735 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368739 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368741 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368744 2571 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368747 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 12:45:49.374597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368749 2571 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368752 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368755 2571 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368758 2571 flags.go:64] FLAG: --kube-reserved="" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368760 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368764 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368767 2571 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368770 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368772 2571 flags.go:64] FLAG: --lock-file="" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368775 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368777 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368780 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368785 2571 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368788 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368790 2571 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368793 2571 flags.go:64] FLAG: --logging-format="text" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368796 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368799 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368801 2571 flags.go:64] FLAG: --manifest-url="" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368804 2571 flags.go:64] FLAG: --manifest-url-header="" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368808 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368810 2571 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368814 2571 flags.go:64] FLAG: --max-pods="110" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368816 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368820 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 12:45:49.375151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368822 2571 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368825 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368828 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368831 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368834 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368840 2571 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368843 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368846 2571 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368848 2571 flags.go:64] FLAG: --pod-cidr="" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368851 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368856 2571 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368859 2571 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368862 2571 flags.go:64] FLAG: --pods-per-core="0" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368865 2571 flags.go:64] FLAG: --port="10250" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368868 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368871 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c64568b04e026bf7" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368874 2571 flags.go:64] FLAG: --qos-reserved="" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368877 2571 flags.go:64] FLAG: --read-only-port="10255" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368879 2571 flags.go:64] FLAG: --register-node="true" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368882 2571 flags.go:64] FLAG: --register-with-taints="" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368885 2571 flags.go:64] FLAG: --registry-burst="10" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368888 2571 flags.go:64] FLAG: --registry-qps="5" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368891 2571 flags.go:64] FLAG: --reserved-cpus="" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368894 2571 flags.go:64] FLAG: --reserved-memory="" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368897 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 12:45:49.375985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368900 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368903 2571 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368905 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368908 2571 flags.go:64] FLAG: --runonce="false" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368911 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368913 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368916 2571 flags.go:64] FLAG: --seccomp-default="false" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368919 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368921 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368924 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368927 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368930 2571 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368933 2571 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368936 2571 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368939 2571 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368941 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368944 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368946 2571 flags.go:64] FLAG: --system-cgroups="" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368949 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368954 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368957 2571 flags.go:64] FLAG: --tls-cert-file="" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368960 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368964 2571 flags.go:64] FLAG: --tls-min-version="" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368966 2571 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368969 2571 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 12:45:49.376739 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368971 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368974 2571 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368977 2571 flags.go:64] FLAG: --v="2" Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368980 2571 flags.go:64] FLAG: --version="false" Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368984 2571 flags.go:64] FLAG: --vmodule="" Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368987 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.368990 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369087 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369091 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369093 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369096 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369099 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369102 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369109 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369111 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369114 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369116 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369118 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369121 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369123 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369126 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 12:45:49.377332 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369129 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369131 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369134 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369136 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369140 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369143 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369146 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369149 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369151 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369154 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369156 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369158 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369161 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369163 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369165 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369168 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369170 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369173 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369175 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369177 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 12:45:49.377803 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369179 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369181 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369184 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369186 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369190 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369192 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369194 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369196 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369199 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369201 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369204 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369212 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369215 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369217 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369220 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369222 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369224 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369226 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369230 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369232 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 12:45:49.378315 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369235 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369237 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369239 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369242 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369244 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369246 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369249 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369251 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369255 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369258 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369260 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369263 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369265 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369267 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369270 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369272 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369276 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369278 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369285 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 12:45:49.378770 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369297 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369300 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369302 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369305 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369307 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369309 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369312 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369315 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369318 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369321 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369323 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369327 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369329 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369331 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369334 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369336 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369338 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369341 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369343 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 12:45:49.379187 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369346 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369348 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369350 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369353 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369355 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369357 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.369360 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.369821 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.377201 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.377223 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.377245 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.377254 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377320 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377325 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377328 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 12:45:49.379629 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377331 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377334 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377337 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377340 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377343 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377346 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377348 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377351 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377353 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377358 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377361 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377364 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377366 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377369 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377371 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377374 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377376 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377378 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377382 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 12:45:49.379967 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377386 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377388 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377390 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377393 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377395 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377397 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377400 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377403 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377405 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377408 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377410 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377413 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377415 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377417 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377419 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377422 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377424 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377427 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377429 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377432 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 12:45:49.380474 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377434 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377436 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377439 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377441 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377444 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377447 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377449 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377451 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377453 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377456 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377458 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377461 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377463 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377465 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377468 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377470 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377473 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377475 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377477 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377480 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 12:45:49.380909 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377483 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377485 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377488 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377491 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377494 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377497 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377499 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377502 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377504 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377506 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377508 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377511 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377513 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377515 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377517 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377520 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377522 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377525 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 12:45:49.381356 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377527 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377530 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377533 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377535 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377537 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377540 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377542 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377544 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377547 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377549 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377551 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377554 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377556 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377559 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377562 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377564 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377566 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377569 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 12:45:49.381739 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377571 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.377576 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377673 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377677 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377679 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377682 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377684 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377687 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377690 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377692 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377695 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377697 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377699 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377702 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377705 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 12:45:49.382153 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377707 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377710 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377712 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377714 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377716 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377719 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377721 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377723 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377726 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377728 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377730 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377732 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377735 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377738 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377741 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377743 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377746 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377748 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377751 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377753 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 12:45:49.382536 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377756 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377759 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377762 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377765 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377768 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377770 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377773 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377775 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377777 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377780 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377783 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377786 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377789 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377791 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377794 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377796 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377798 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377800 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377803 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 12:45:49.382962 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377805 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377808 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377810 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377813 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377816 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377818 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377821 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377823 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377825 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377828 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377830 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377833 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377835 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377837 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377839 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377842 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377844 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377846 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377848 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377850 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 12:45:49.383440 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377853 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377855 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377857 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377859 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377862 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377864 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377867 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377869 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377871 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377874 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377876 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377878 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377880 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377883 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377885 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377887 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377890 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377892 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377895 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 12:45:49.383875 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377898 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377901 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377904 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377907 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377910 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377912 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377915 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:49.377917 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.377922 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.378023 2571 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.380787 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.381959 2571 server.go:1019] "Starting client certificate rotation" Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.382054 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 12:45:49.384346 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.382080 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 12:45:49.384886 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.384872 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 12:45:49.386399 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.386384 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 12:45:49.395004 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.394986 2571 log.go:25] "Validated CRI v1 runtime API" Jul 10 12:45:49.399105 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.399088 2571 log.go:25] "Validated CRI v1 image API" Jul 10 12:45:49.399825 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.399811 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 12:45:49.402034 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.402018 2571 fs.go:135] Filesystem UUIDs: map[571ca475-e8c5-401c-984a-c35784db2df5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d3c64a6d-525a-487e-ba58-343ea3a34367:/dev/nvme0n1p4] Jul 10 12:45:49.402089 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.402034 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 12:45:49.406201 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.406105 2571 manager.go:216] Machine: {Timestamp:2026-07-10 12:45:49.405648546 +0000 UTC m=+0.291409151 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102760 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26f0eb8ad7df5bc30e9ab6848022b2 SystemUUID:ec26f0eb-8ad7-df5b-c30e-9ab6848022b2 BootID:9c61d617-e80d-433e-883e-c60759288b96 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:46:2f:63:38:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:46:2f:63:38:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:96:b2:dc:4d:18 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 12:45:49.406201 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.406192 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 12:45:49.406337 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.406265 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 12:45:49.408816 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.408792 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 12:45:49.408960 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.408826 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-162.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 12:45:49.409383 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.409369 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 12:45:49.409487 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.409476 2571 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 12:45:49.409520 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.409489 2571 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 12:45:49.409520 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.409496 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 12:45:49.410172 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.410162 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 12:45:49.411272 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.411262 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 12:45:49.414571 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.414561 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 10 12:45:49.414698 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.414689 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 12:45:49.414732 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.414715 2571 kubelet.go:486] "Attempting to sync node with API server" Jul 10 12:45:49.414732 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.414729 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 12:45:49.414778 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.414744 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 12:45:49.414778 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.414753 2571 kubelet.go:398] "Adding apiserver pod source" Jul 10 12:45:49.414778 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.414761 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 12:45:49.416626 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.416606 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 12:45:49.416702 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.416638 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 12:45:49.419553 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.419535 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 12:45:49.421166 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421151 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 12:45:49.421240 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421179 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 12:45:49.421854 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421844 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 12:45:49.421908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421861 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 12:45:49.421908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421867 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 12:45:49.421908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421872 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 12:45:49.421908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421879 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 12:45:49.421908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421885 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 12:45:49.421908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421893 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 12:45:49.421908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421897 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 12:45:49.421908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421903 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 12:45:49.421908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421908 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 12:45:49.422115 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421916 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 12:45:49.422115 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.421920 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 12:45:49.422590 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.422581 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 12:45:49.422590 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.422590 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 12:45:49.425655 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.425641 2571 server.go:1268] "Started kubelet" Jul 10 12:45:49.425777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.425716 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 12:45:49.425841 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.425796 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 12:45:49.425944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.425897 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 12:45:49.426398 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.426375 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-162.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 12:45:49.426491 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.426443 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 12:45:49.426546 ip-10-0-143-162 systemd[1]: Started Kubernetes Kubelet. Jul 10 12:45:49.426641 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.426558 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-162.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 12:45:49.426719 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.426703 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 12:45:49.427868 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.427855 2571 server.go:310] "Adding debug handlers to kubelet server" Jul 10 12:45:49.431609 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.431590 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 12:45:49.431683 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.431606 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 12:45:49.431729 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.431682 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:49.432212 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.432198 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 12:45:49.432349 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.432306 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 12:45:49.432418 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.432199 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 12:45:49.432418 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.432412 2571 reconstruct.go:97] "Volume reconstruction finished" Jul 10 12:45:49.432418 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.432419 2571 reconciler.go:29] "Reconciler: start to sync state" Jul 10 12:45:49.432860 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.432823 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 12:45:49.432941 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.432900 2571 factory.go:55] Registering systemd factory Jul 10 12:45:49.432941 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.432912 2571 factory.go:223] Registration of the systemd container factory successfully Jul 10 12:45:49.433070 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.433059 2571 factory.go:153] Registering CRI-O factory Jul 10 12:45:49.433070 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.433069 2571 factory.go:223] Registration of the crio container factory successfully Jul 10 12:45:49.433167 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.433145 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 12:45:49.433219 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.433176 2571 factory.go:103] Registering Raw factory Jul 10 12:45:49.433219 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.433188 2571 manager.go:1196] Started watching for new ooms in manager Jul 10 12:45:49.433454 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.432518 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-162.ec2.internal.18c0ee6035cd0f56 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-162.ec2.internal,UID:ip-10-0-143-162.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-162.ec2.internal,},FirstTimestamp:2026-07-10 12:45:49.425618774 +0000 UTC m=+0.311379376,LastTimestamp:2026-07-10 12:45:49.425618774 +0000 UTC m=+0.311379376,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-162.ec2.internal,}" Jul 10 12:45:49.433722 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.433707 2571 manager.go:349] Starting recovery of all containers Jul 10 12:45:49.439618 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.439477 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-162.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 12:45:49.439618 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.439511 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 12:45:49.439747 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.439513 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 12:45:49.445571 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.445553 2571 manager.go:354] Recovery completed Jul 10 12:45:49.449547 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.449535 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 12:45:49.451118 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.451098 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientMemory" Jul 10 12:45:49.451168 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.451129 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 12:45:49.451168 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.451138 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientPID" Jul 10 12:45:49.451614 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.451603 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 12:45:49.451614 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.451613 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 12:45:49.451696 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.451626 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 10 12:45:49.451696 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.451671 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 12:45:49.452524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.452509 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientMemory" Jul 10 12:45:49.452602 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.452533 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 12:45:49.452602 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.452544 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientPID" Jul 10 12:45:49.453030 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.452961 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-162.ec2.internal.18c0ee6037522693 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-162.ec2.internal,UID:ip-10-0-143-162.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-162.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-162.ec2.internal,},FirstTimestamp:2026-07-10 12:45:49.451118227 +0000 UTC m=+0.336878829,LastTimestamp:2026-07-10 12:45:49.451118227 +0000 UTC m=+0.336878829,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-162.ec2.internal,}" Jul 10 12:45:49.453600 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.453589 2571 policy_none.go:49] "None policy: Start" Jul 10 12:45:49.453641 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.453603 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 12:45:49.453641 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.453612 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 12:45:49.455357 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.455346 2571 policy_none.go:47] "Start" Jul 10 12:45:49.462590 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.462574 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5szd5" Jul 10 12:45:49.463089 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.463036 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-162.ec2.internal.18c0ee60375261e2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-162.ec2.internal,UID:ip-10-0-143-162.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-162.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-162.ec2.internal,},FirstTimestamp:2026-07-10 12:45:49.45113341 +0000 UTC m=+0.336894014,LastTimestamp:2026-07-10 12:45:49.45113341 +0000 UTC m=+0.336894014,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-162.ec2.internal,}" Jul 10 12:45:49.472643 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.472627 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5szd5" Jul 10 12:45:49.476082 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.476020 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-162.ec2.internal.18c0ee60375281fa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-162.ec2.internal,UID:ip-10-0-143-162.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-143-162.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-143-162.ec2.internal,},FirstTimestamp:2026-07-10 12:45:49.451141626 +0000 UTC m=+0.336902229,LastTimestamp:2026-07-10 12:45:49.451141626 +0000 UTC m=+0.336902229,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-162.ec2.internal,}" Jul 10 12:45:49.490826 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.490814 2571 manager.go:337] "Starting Device Plugin manager" Jul 10 12:45:49.490897 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.490840 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 12:45:49.490897 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.490848 2571 server.go:87] "Starting device plugin registration server" Jul 10 12:45:49.491093 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.491083 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 12:45:49.491126 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.491096 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 12:45:49.491206 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.491192 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 12:45:49.491263 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.491256 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 12:45:49.491325 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.491264 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 12:45:49.491851 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.491825 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 12:45:49.491931 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.491875 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:49.528545 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.528524 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 12:45:49.536434 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.528562 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 12:45:49.536434 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.528581 2571 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 12:45:49.536434 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.528612 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 12:45:49.536434 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.530709 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 12:45:49.591649 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.591628 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 12:45:49.592546 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.592531 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientMemory" Jul 10 12:45:49.592626 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.592564 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 12:45:49.592626 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.592575 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientPID" Jul 10 12:45:49.592626 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.592595 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.605494 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.605478 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.605536 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.605499 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-162.ec2.internal\": node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:49.629727 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.629675 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal"] Jul 10 12:45:49.629783 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.629733 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 12:45:49.630421 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.630400 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientMemory" Jul 10 12:45:49.630500 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.630428 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 12:45:49.630500 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.630443 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientPID" Jul 10 12:45:49.632482 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.632469 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 12:45:49.632547 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.632532 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.632582 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.632562 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 12:45:49.632638 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.632618 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a52decb2df7daf27b74a4f34aa1fb98-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal\" (UID: \"6a52decb2df7daf27b74a4f34aa1fb98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.633119 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.633104 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientMemory" Jul 10 12:45:49.633181 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.633128 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 12:45:49.633181 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.633141 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientPID" Jul 10 12:45:49.633181 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.633141 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientMemory" Jul 10 12:45:49.633181 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.633167 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 12:45:49.633181 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.633178 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientPID" Jul 10 12:45:49.635261 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.635245 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.635366 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.635268 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 12:45:49.635911 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.635891 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientMemory" Jul 10 12:45:49.635981 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.635918 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 12:45:49.635981 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.635931 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientPID" Jul 10 12:45:49.636079 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.635999 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:49.653794 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.653776 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-162.ec2.internal\" not found" node="ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.656874 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.656859 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-162.ec2.internal\" not found" node="ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.733397 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.733373 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a52decb2df7daf27b74a4f34aa1fb98-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal\" (UID: \"6a52decb2df7daf27b74a4f34aa1fb98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.733501 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.733401 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a52decb2df7daf27b74a4f34aa1fb98-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal\" (UID: \"6a52decb2df7daf27b74a4f34aa1fb98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.733501 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.733416 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1078cbfb33567ae55c66f9d84943893-config\") pod \"kube-apiserver-proxy-ip-10-0-143-162.ec2.internal\" (UID: \"d1078cbfb33567ae55c66f9d84943893\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.733501 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.733460 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a52decb2df7daf27b74a4f34aa1fb98-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal\" (UID: \"6a52decb2df7daf27b74a4f34aa1fb98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.736442 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.736427 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:49.833961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.833927 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a52decb2df7daf27b74a4f34aa1fb98-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal\" (UID: \"6a52decb2df7daf27b74a4f34aa1fb98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.833961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.833958 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1078cbfb33567ae55c66f9d84943893-config\") pod \"kube-apiserver-proxy-ip-10-0-143-162.ec2.internal\" (UID: \"d1078cbfb33567ae55c66f9d84943893\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.834131 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.833993 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1078cbfb33567ae55c66f9d84943893-config\") pod \"kube-apiserver-proxy-ip-10-0-143-162.ec2.internal\" (UID: \"d1078cbfb33567ae55c66f9d84943893\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.834131 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.834005 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a52decb2df7daf27b74a4f34aa1fb98-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal\" (UID: \"6a52decb2df7daf27b74a4f34aa1fb98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.837030 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.837009 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:49.937698 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:49.937634 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:49.955786 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.955763 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" Jul 10 12:45:49.960154 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:49.960124 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal" Jul 10 12:45:50.038685 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:50.038649 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:50.139031 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:50.139007 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:50.239539 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:50.239479 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:50.283162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.283138 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 12:45:50.340138 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:50.340114 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:50.381269 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.381246 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 12:45:50.381687 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.381389 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 12:45:50.381687 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.381404 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 12:45:50.440605 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:50.440580 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-162.ec2.internal\" not found" Jul 10 12:45:50.479123 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.479090 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 12:40:49 +0000 UTC" deadline="2028-01-03 21:28:41.611338174 +0000 UTC" Jul 10 12:45:50.479123 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.479116 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13016h42m51.132224692s" Jul 10 12:45:50.508348 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.508329 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 12:45:50.509362 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.509343 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientMemory" Jul 10 12:45:50.509458 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.509373 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 12:45:50.509458 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.509383 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeHasSufficientPID" Jul 10 12:45:50.515512 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.515490 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 12:45:50.518399 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:50.518378 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1078cbfb33567ae55c66f9d84943893.slice/crio-efc1071d545df1e77c5818f700049448a8e11cf5dc0f29df0b9d43c39ace2ef9 WatchSource:0}: Error finding container efc1071d545df1e77c5818f700049448a8e11cf5dc0f29df0b9d43c39ace2ef9: Status 404 returned error can't find the container with id efc1071d545df1e77c5818f700049448a8e11cf5dc0f29df0b9d43c39ace2ef9 Jul 10 12:45:50.518760 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:50.518742 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a52decb2df7daf27b74a4f34aa1fb98.slice/crio-fe02f858ecb150a16b9ebb915b127a461de688606b2b346c23c233913add5041 WatchSource:0}: Error finding container fe02f858ecb150a16b9ebb915b127a461de688606b2b346c23c233913add5041: Status 404 returned error can't find the container with id fe02f858ecb150a16b9ebb915b127a461de688606b2b346c23c233913add5041 Jul 10 12:45:50.524360 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.524348 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 12:45:50.531871 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.531855 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" Jul 10 12:45:50.532041 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.532014 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal" event={"ID":"d1078cbfb33567ae55c66f9d84943893","Type":"ContainerStarted","Data":"efc1071d545df1e77c5818f700049448a8e11cf5dc0f29df0b9d43c39ace2ef9"} Jul 10 12:45:50.532965 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.532946 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" event={"ID":"6a52decb2df7daf27b74a4f34aa1fb98","Type":"ContainerStarted","Data":"fe02f858ecb150a16b9ebb915b127a461de688606b2b346c23c233913add5041"} Jul 10 12:45:50.543173 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.543155 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 12:45:50.543949 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.543939 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal" Jul 10 12:45:50.549456 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.549446 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 12:45:50.631661 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:50.631641 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 12:45:51.415912 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.415884 2571 apiserver.go:52] "Watching apiserver" Jul 10 12:45:51.424769 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.424747 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 12:45:51.425949 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.425926 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-6hl8t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal","openshift-multus/network-metrics-daemon-frf69","openshift-network-diagnostics/network-check-target-572wk","openshift-network-operator/iptables-alerter-pt8lv","openshift-ovn-kubernetes/ovnkube-node-rhz66","kube-system/global-pull-secret-syncer-8ln2b","kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c","openshift-cluster-node-tuning-operator/tuned-tmnx8","openshift-image-registry/node-ca-84kdj","openshift-multus/multus-additional-cni-plugins-ql2kp","openshift-multus/multus-bwb2c","kube-system/konnectivity-agent-gmfsc"] Jul 10 12:45:51.428345 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.428322 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:51.428446 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.428405 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:45:51.430542 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.430515 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.431761 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.431744 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 12:45:51.434135 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.433626 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 12:45:51.434135 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.433681 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 12:45:51.434135 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.433852 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:51.434135 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.433875 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 12:45:51.434135 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.433906 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:45:51.434135 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.433994 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tpt8g\"" Jul 10 12:45:51.434135 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.434017 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 12:45:51.435407 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.435166 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:51.435407 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.435236 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:45:51.437399 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.437380 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.440133 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.440040 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.440133 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.440072 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 12:45:51.440756 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.440566 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 12:45:51.440756 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.440570 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 12:45:51.440756 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.440652 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bdw9x\"" Jul 10 12:45:51.442777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.442760 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.443106 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.442957 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 12:45:51.443191 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443118 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-cnibin\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443191 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443145 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-var-lib-cni-bin\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443333 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443190 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-var-lib-kubelet\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443333 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443216 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zgvc\" (UniqueName: \"kubernetes.io/projected/f2b3b569-5224-4053-81bb-9ab20a2a397f-kube-api-access-5zgvc\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443333 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443263 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:51.443333 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443307 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-os-release\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443341 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-socket-dir-parent\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443365 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-run-netns\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443388 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-var-lib-cni-multus\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443438 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x4v4\" (UniqueName: \"kubernetes.io/projected/daa28724-34bf-4c69-9bec-fcee00bc456f-kube-api-access-9x4v4\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:51.443524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443463 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea5247d2-85e6-4959-9479-a6eea1212a2d-dbus\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:51.443524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443485 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:51.443524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443505 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-system-cni-dir\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443825 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443529 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-conf-dir\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443825 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443551 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:51.443825 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443585 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-cni-dir\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443825 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443606 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2b3b569-5224-4053-81bb-9ab20a2a397f-cni-binary-copy\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.443825 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443623 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k7w59\"" Jul 10 12:45:51.443825 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.443641 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-run-k8s-cni-cncf-io\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.444479 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.444455 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-hostroot\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.444558 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.444507 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-daemon-config\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.444558 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.444539 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-run-multus-certs\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.444648 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.444567 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-etc-kubernetes\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.444648 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.444590 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea5247d2-85e6-4959-9479-a6eea1212a2d-kubelet-config\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:51.444875 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.444854 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.444986 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.444968 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 12:45:51.445189 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.445167 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 12:45:51.445843 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.445805 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 12:45:51.446406 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.446389 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 12:45:51.448151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.447064 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 12:45:51.448151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.447268 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 12:45:51.448151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.447499 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 12:45:51.448615 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.448550 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 12:45:51.449726 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.449450 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-49zhw\"" Jul 10 12:45:51.449726 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.449542 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 12:45:51.450280 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.450248 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:45:51.451072 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.451049 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 12:45:51.451757 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.451737 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ps5h7\"" Jul 10 12:45:51.451854 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.451813 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 12:45:51.453256 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.453237 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.456700 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.456683 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 12:45:51.457493 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.457479 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 12:45:51.457932 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.457531 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nqwr4\"" Jul 10 12:45:51.458014 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.457595 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-clqj7\"" Jul 10 12:45:51.458070 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.457614 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 12:45:51.458070 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.457637 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 12:45:51.458155 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.457770 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.458477 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.458459 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.461771 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.460699 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 12:45:51.461771 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.461062 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 12:45:51.461771 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.461170 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 12:45:51.461771 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.461237 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 12:45:51.461771 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.461404 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-p4blf\"" Jul 10 12:45:51.461771 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.461421 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gbk2h\"" Jul 10 12:45:51.461771 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.461545 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 12:45:51.507585 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.507555 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l2dzc" Jul 10 12:45:51.516415 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.516393 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l2dzc" Jul 10 12:45:51.533890 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.533873 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 12:45:51.545746 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545722 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83fab052-d823-4c52-bf03-26323aba154d-agent-certs\") pod \"konnectivity-agent-gmfsc\" (UID: \"83fab052-d823-4c52-bf03-26323aba154d\") " pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:45:51.545848 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545759 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-lib-modules\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.545848 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545783 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-etc-openvswitch\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.545848 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545810 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2b3b569-5224-4053-81bb-9ab20a2a397f-cni-binary-copy\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.545848 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545834 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-etc-kubernetes\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.546045 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545855 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4aeb355e-62c8-4c45-9b4a-795ab74cfe42-hosts-file\") pod \"node-resolver-6hl8t\" (UID: \"4aeb355e-62c8-4c45-9b4a-795ab74cfe42\") " pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.546045 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545880 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-system-cni-dir\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.546045 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545904 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-sysconfig\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.546045 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545939 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11d98db9-882d-4480-b299-36d20927e585-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.546045 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545962 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11d98db9-882d-4480-b299-36d20927e585-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.546045 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.545985 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-cni-bin\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.546045 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546006 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-cni-netd\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.546045 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546028 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-socket-dir-parent\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546049 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-var-lib-cni-multus\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546070 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-var-lib-kubelet\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546092 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-run-ovn-kubernetes\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546136 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/27ee70dd-0218-4c07-a51f-68e69418cbdf-ovnkube-config\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546168 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546191 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-os-release\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546213 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-run-systemd\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546233 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-hostroot\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546261 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546317 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-registration-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546333 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-host\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546347 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsbkf\" (UniqueName: \"kubernetes.io/projected/390a4dd4-04cc-4db6-93e5-33ee6222c93e-kube-api-access-qsbkf\") pod \"node-ca-84kdj\" (UID: \"390a4dd4-04cc-4db6-93e5-33ee6222c93e\") " pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546351 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-etc-kubernetes\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546378 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/390a4dd4-04cc-4db6-93e5-33ee6222c93e-host\") pod \"node-ca-84kdj\" (UID: \"390a4dd4-04cc-4db6-93e5-33ee6222c93e\") " pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546396 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea5247d2-85e6-4959-9479-a6eea1212a2d-kubelet-config\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546412 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-cnibin\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.546437 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546419 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2b3b569-5224-4053-81bb-9ab20a2a397f-cni-binary-copy\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546437 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zgvc\" (UniqueName: \"kubernetes.io/projected/f2b3b569-5224-4053-81bb-9ab20a2a397f-kube-api-access-5zgvc\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546467 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546480 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4aeb355e-62c8-4c45-9b4a-795ab74cfe42-tmp-dir\") pod \"node-resolver-6hl8t\" (UID: \"4aeb355e-62c8-4c45-9b4a-795ab74cfe42\") " pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546498 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546523 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-os-release\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546537 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-socket-dir-parent\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546543 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-run-netns\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546576 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-run-netns\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546590 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x4v4\" (UniqueName: \"kubernetes.io/projected/daa28724-34bf-4c69-9bec-fcee00bc456f-kube-api-access-9x4v4\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546626 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw2hb\" (UniqueName: \"kubernetes.io/projected/11d98db9-882d-4480-b299-36d20927e585-kube-api-access-mw2hb\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546656 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn7zl\" (UniqueName: \"kubernetes.io/projected/dfd3af5f-f19f-48a0-97d2-cd472064b05c-kube-api-access-bn7zl\") pod \"iptables-alerter-pt8lv\" (UID: \"dfd3af5f-f19f-48a0-97d2-cd472064b05c\") " pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546683 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/27ee70dd-0218-4c07-a51f-68e69418cbdf-env-overrides\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546715 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-system-cni-dir\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546742 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-device-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546768 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-slash\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546795 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-run-openvswitch\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.547184 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546820 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-node-log\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.546878 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/27ee70dd-0218-4c07-a51f-68e69418cbdf-ovn-node-metrics-cert\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547042 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-cni-dir\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547107 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-os-release\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547158 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-cni-dir\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.547170 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.547227 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs podName:daa28724-34bf-4c69-9bec-fcee00bc456f nodeName:}" failed. No retries permitted until 2026-07-10 12:45:52.04720888 +0000 UTC m=+2.932969475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs") pod "network-metrics-daemon-frf69" (UID: "daa28724-34bf-4c69-9bec-fcee00bc456f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547272 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-cnibin\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547344 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-hostroot\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547379 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-var-lib-cni-multus\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547387 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ea5247d2-85e6-4959-9479-a6eea1212a2d-kubelet-config\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547408 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-run-k8s-cni-cncf-io\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547435 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-daemon-config\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547457 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-run-multus-certs\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547497 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-sysctl-d\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547521 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-sysctl-conf\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547544 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-var-lib-openvswitch\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.547903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547581 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-var-lib-kubelet\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.548693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547608 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s55k5\" (UniqueName: \"kubernetes.io/projected/565c20e1-8a81-4655-9155-9de32af9e470-kube-api-access-s55k5\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.548693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547632 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-systemd\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.548693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547678 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-run-multus-certs\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.548693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547711 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-run-k8s-cni-cncf-io\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.548693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547750 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-system-cni-dir\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.548693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547787 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/390a4dd4-04cc-4db6-93e5-33ee6222c93e-serviceca\") pod \"node-ca-84kdj\" (UID: \"390a4dd4-04cc-4db6-93e5-33ee6222c93e\") " pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.548693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547812 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-cnibin\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.548693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547836 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-run-ovn\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.548693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.547859 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-systemd-units\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.548264 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-daemon-config\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550401 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-var-lib-kubelet\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550448 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/27ee70dd-0218-4c07-a51f-68e69418cbdf-ovnkube-script-lib\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550482 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sblrw\" (UniqueName: \"kubernetes.io/projected/27ee70dd-0218-4c07-a51f-68e69418cbdf-kube-api-access-sblrw\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550510 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-socket-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550534 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-sys-fs\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550564 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83fab052-d823-4c52-bf03-26323aba154d-konnectivity-ca\") pod \"konnectivity-agent-gmfsc\" (UID: \"83fab052-d823-4c52-bf03-26323aba154d\") " pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550612 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-sys\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550641 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-run-netns\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550693 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-log-socket\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.550777 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550733 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.551343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550826 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11d98db9-882d-4480-b299-36d20927e585-cni-binary-copy\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.551343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550875 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dfd3af5f-f19f-48a0-97d2-cd472064b05c-host-slash\") pod \"iptables-alerter-pt8lv\" (UID: \"dfd3af5f-f19f-48a0-97d2-cd472064b05c\") " pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.551343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550909 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-var-lib-cni-bin\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.551343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.550971 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-etc-selinux\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.551343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551001 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-run\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.551343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551026 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11de507a-38ab-47e2-bafe-c84044511a7a-tmp\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.551343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551133 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dfd3af5f-f19f-48a0-97d2-cd472064b05c-iptables-alerter-script\") pod \"iptables-alerter-pt8lv\" (UID: \"dfd3af5f-f19f-48a0-97d2-cd472064b05c\") " pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.551645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551382 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwrc7\" (UniqueName: \"kubernetes.io/projected/4aeb355e-62c8-4c45-9b4a-795ab74cfe42-kube-api-access-lwrc7\") pod \"node-resolver-6hl8t\" (UID: \"4aeb355e-62c8-4c45-9b4a-795ab74cfe42\") " pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.551645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551415 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-modprobe-d\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.551645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551446 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-kubernetes\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.551645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551472 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11de507a-38ab-47e2-bafe-c84044511a7a-etc-tuned\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.551645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551502 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-kubelet\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.551645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551532 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8tq5\" (UniqueName: \"kubernetes.io/projected/11de507a-38ab-47e2-bafe-c84044511a7a-kube-api-access-b8tq5\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.551645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551570 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea5247d2-85e6-4959-9479-a6eea1212a2d-dbus\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:51.551645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.551604 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:51.552196 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.552046 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-conf-dir\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.552196 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.552149 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-multus-conf-dir\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.552196 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.552132 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ea5247d2-85e6-4959-9479-a6eea1212a2d-dbus\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:51.552424 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.552236 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:51.552424 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.552319 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret podName:ea5247d2-85e6-4959-9479-a6eea1212a2d nodeName:}" failed. No retries permitted until 2026-07-10 12:45:52.052302066 +0000 UTC m=+2.938062674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret") pod "global-pull-secret-syncer-8ln2b" (UID: "ea5247d2-85e6-4959-9479-a6eea1212a2d") : object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:51.552424 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.552371 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2b3b569-5224-4053-81bb-9ab20a2a397f-host-var-lib-cni-bin\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.553850 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.553827 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 12:45:51.554955 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.554935 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 12:45:51.554955 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.554958 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 12:45:51.555092 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.554970 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kmldp for pod openshift-network-diagnostics/network-check-target-572wk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:51.555092 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:51.555023 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp podName:eb142d03-253f-4af6-97a8-80adc3e17656 nodeName:}" failed. No retries permitted until 2026-07-10 12:45:52.055010548 +0000 UTC m=+2.940771141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kmldp" (UniqueName: "kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp") pod "network-check-target-572wk" (UID: "eb142d03-253f-4af6-97a8-80adc3e17656") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:51.558608 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.558549 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x4v4\" (UniqueName: \"kubernetes.io/projected/daa28724-34bf-4c69-9bec-fcee00bc456f-kube-api-access-9x4v4\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:51.559474 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.559177 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zgvc\" (UniqueName: \"kubernetes.io/projected/f2b3b569-5224-4053-81bb-9ab20a2a397f-kube-api-access-5zgvc\") pod \"multus-bwb2c\" (UID: \"f2b3b569-5224-4053-81bb-9ab20a2a397f\") " pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.652762 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652714 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11de507a-38ab-47e2-bafe-c84044511a7a-tmp\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.652762 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652759 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dfd3af5f-f19f-48a0-97d2-cd472064b05c-iptables-alerter-script\") pod \"iptables-alerter-pt8lv\" (UID: \"dfd3af5f-f19f-48a0-97d2-cd472064b05c\") " pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.652979 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652786 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwrc7\" (UniqueName: \"kubernetes.io/projected/4aeb355e-62c8-4c45-9b4a-795ab74cfe42-kube-api-access-lwrc7\") pod \"node-resolver-6hl8t\" (UID: \"4aeb355e-62c8-4c45-9b4a-795ab74cfe42\") " pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.652979 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652809 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-modprobe-d\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.652979 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652833 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-kubernetes\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.652979 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652854 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11de507a-38ab-47e2-bafe-c84044511a7a-etc-tuned\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.652979 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652876 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-kubelet\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.652979 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652899 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8tq5\" (UniqueName: \"kubernetes.io/projected/11de507a-38ab-47e2-bafe-c84044511a7a-kube-api-access-b8tq5\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.652979 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652937 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83fab052-d823-4c52-bf03-26323aba154d-agent-certs\") pod \"konnectivity-agent-gmfsc\" (UID: \"83fab052-d823-4c52-bf03-26323aba154d\") " pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:45:51.652979 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652960 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-lib-modules\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.652994 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-etc-openvswitch\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653019 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4aeb355e-62c8-4c45-9b4a-795ab74cfe42-hosts-file\") pod \"node-resolver-6hl8t\" (UID: \"4aeb355e-62c8-4c45-9b4a-795ab74cfe42\") " pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653042 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-system-cni-dir\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653064 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-sysconfig\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653086 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11d98db9-882d-4480-b299-36d20927e585-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653111 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11d98db9-882d-4480-b299-36d20927e585-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653198 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-system-cni-dir\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653260 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-etc-openvswitch\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653257 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-lib-modules\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.653377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653348 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-sysconfig\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.653848 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653396 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4aeb355e-62c8-4c45-9b4a-795ab74cfe42-hosts-file\") pod \"node-resolver-6hl8t\" (UID: \"4aeb355e-62c8-4c45-9b4a-795ab74cfe42\") " pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.653848 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653752 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-kubernetes\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.653949 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653892 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-modprobe-d\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.653949 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653919 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dfd3af5f-f19f-48a0-97d2-cd472064b05c-iptables-alerter-script\") pod \"iptables-alerter-pt8lv\" (UID: \"dfd3af5f-f19f-48a0-97d2-cd472064b05c\") " pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.653949 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653943 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-kubelet\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.654094 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653969 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-cni-bin\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.654094 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.653998 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-cni-netd\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.654094 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654024 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-var-lib-kubelet\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.654094 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654050 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-run-ovn-kubernetes\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.654094 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654073 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/27ee70dd-0218-4c07-a51f-68e69418cbdf-ovnkube-config\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.654359 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654130 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-os-release\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.654359 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654154 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-run-systemd\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.654359 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654179 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.654359 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654203 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-registration-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.654359 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654228 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-host\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.654359 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654235 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-run-ovn-kubernetes\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.654359 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654251 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsbkf\" (UniqueName: \"kubernetes.io/projected/390a4dd4-04cc-4db6-93e5-33ee6222c93e-kube-api-access-qsbkf\") pod \"node-ca-84kdj\" (UID: \"390a4dd4-04cc-4db6-93e5-33ee6222c93e\") " pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654383 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-registration-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654438 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654491 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-host\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654523 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/390a4dd4-04cc-4db6-93e5-33ee6222c93e-host\") pod \"node-ca-84kdj\" (UID: \"390a4dd4-04cc-4db6-93e5-33ee6222c93e\") " pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654564 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4aeb355e-62c8-4c45-9b4a-795ab74cfe42-tmp-dir\") pod \"node-resolver-6hl8t\" (UID: \"4aeb355e-62c8-4c45-9b4a-795ab74cfe42\") " pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654586 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654615 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw2hb\" (UniqueName: \"kubernetes.io/projected/11d98db9-882d-4480-b299-36d20927e585-kube-api-access-mw2hb\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654656 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn7zl\" (UniqueName: \"kubernetes.io/projected/dfd3af5f-f19f-48a0-97d2-cd472064b05c-kube-api-access-bn7zl\") pod \"iptables-alerter-pt8lv\" (UID: \"dfd3af5f-f19f-48a0-97d2-cd472064b05c\") " pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/27ee70dd-0218-4c07-a51f-68e69418cbdf-env-overrides\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.654721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654707 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-device-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654730 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-slash\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654755 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-run-openvswitch\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654779 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-node-log\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654802 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/27ee70dd-0218-4c07-a51f-68e69418cbdf-ovn-node-metrics-cert\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654831 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-sysctl-d\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654855 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-sysctl-conf\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654879 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-var-lib-openvswitch\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654907 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s55k5\" (UniqueName: \"kubernetes.io/projected/565c20e1-8a81-4655-9155-9de32af9e470-kube-api-access-s55k5\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654944 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-systemd\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654967 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/390a4dd4-04cc-4db6-93e5-33ee6222c93e-serviceca\") pod \"node-ca-84kdj\" (UID: \"390a4dd4-04cc-4db6-93e5-33ee6222c93e\") " pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654985 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/11d98db9-882d-4480-b299-36d20927e585-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655040 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-device-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655046 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-cnibin\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655083 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-run-openvswitch\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655088 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-slash\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655131 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-node-log\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.655300 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655208 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-sysctl-d\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.656104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655284 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/27ee70dd-0218-4c07-a51f-68e69418cbdf-env-overrides\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.656104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655343 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-sysctl-conf\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.656104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655385 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-var-lib-openvswitch\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.656104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655624 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-etc-systemd\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.656104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655737 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/27ee70dd-0218-4c07-a51f-68e69418cbdf-ovnkube-config\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.656104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655810 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-os-release\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.656104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.655860 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/390a4dd4-04cc-4db6-93e5-33ee6222c93e-host\") pod \"node-ca-84kdj\" (UID: \"390a4dd4-04cc-4db6-93e5-33ee6222c93e\") " pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.656104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656024 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/390a4dd4-04cc-4db6-93e5-33ee6222c93e-serviceca\") pod \"node-ca-84kdj\" (UID: \"390a4dd4-04cc-4db6-93e5-33ee6222c93e\") " pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656484 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/11d98db9-882d-4480-b299-36d20927e585-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656552 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-cni-netd\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656610 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-cni-bin\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.654992 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-cnibin\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656647 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-run-ovn\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656673 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-systemd-units\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656696 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/27ee70dd-0218-4c07-a51f-68e69418cbdf-ovnkube-script-lib\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656728 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sblrw\" (UniqueName: \"kubernetes.io/projected/27ee70dd-0218-4c07-a51f-68e69418cbdf-kube-api-access-sblrw\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656762 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-socket-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656785 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-sys-fs\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656811 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83fab052-d823-4c52-bf03-26323aba154d-konnectivity-ca\") pod \"konnectivity-agent-gmfsc\" (UID: \"83fab052-d823-4c52-bf03-26323aba154d\") " pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656833 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-sys\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-run-netns\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656878 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-log-socket\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656902 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656894 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11d98db9-882d-4480-b299-36d20927e585-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.658101 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657072 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-run-ovn\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657109 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-systemd-units\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657423 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11d98db9-882d-4480-b299-36d20927e585-cni-binary-copy\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657519 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-var-lib-kubelet\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657562 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-run-systemd\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657575 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/27ee70dd-0218-4c07-a51f-68e69418cbdf-ovnkube-script-lib\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657626 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-host-run-netns\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657628 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-sys\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.656947 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11d98db9-882d-4480-b299-36d20927e585-cni-binary-copy\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657663 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/27ee70dd-0218-4c07-a51f-68e69418cbdf-log-socket\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657672 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dfd3af5f-f19f-48a0-97d2-cd472064b05c-host-slash\") pod \"iptables-alerter-pt8lv\" (UID: \"dfd3af5f-f19f-48a0-97d2-cd472064b05c\") " pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657700 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-etc-selinux\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657723 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-run\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657748 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11de507a-38ab-47e2-bafe-c84044511a7a-tmp\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657770 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-socket-dir\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657792 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11de507a-38ab-47e2-bafe-c84044511a7a-run\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657841 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dfd3af5f-f19f-48a0-97d2-cd472064b05c-host-slash\") pod \"iptables-alerter-pt8lv\" (UID: \"dfd3af5f-f19f-48a0-97d2-cd472064b05c\") " pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.659033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657877 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-etc-selinux\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.659898 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.657919 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/565c20e1-8a81-4655-9155-9de32af9e470-sys-fs\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.659898 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.658065 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83fab052-d823-4c52-bf03-26323aba154d-konnectivity-ca\") pod \"konnectivity-agent-gmfsc\" (UID: \"83fab052-d823-4c52-bf03-26323aba154d\") " pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:45:51.659898 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.658464 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11de507a-38ab-47e2-bafe-c84044511a7a-etc-tuned\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.661313 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.660260 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4aeb355e-62c8-4c45-9b4a-795ab74cfe42-tmp-dir\") pod \"node-resolver-6hl8t\" (UID: \"4aeb355e-62c8-4c45-9b4a-795ab74cfe42\") " pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.661313 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.660277 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/27ee70dd-0218-4c07-a51f-68e69418cbdf-ovn-node-metrics-cert\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.661313 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.660492 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83fab052-d823-4c52-bf03-26323aba154d-agent-certs\") pod \"konnectivity-agent-gmfsc\" (UID: \"83fab052-d823-4c52-bf03-26323aba154d\") " pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:45:51.663654 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.663631 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwrc7\" (UniqueName: \"kubernetes.io/projected/4aeb355e-62c8-4c45-9b4a-795ab74cfe42-kube-api-access-lwrc7\") pod \"node-resolver-6hl8t\" (UID: \"4aeb355e-62c8-4c45-9b4a-795ab74cfe42\") " pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.664804 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.664374 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8tq5\" (UniqueName: \"kubernetes.io/projected/11de507a-38ab-47e2-bafe-c84044511a7a-kube-api-access-b8tq5\") pod \"tuned-tmnx8\" (UID: \"11de507a-38ab-47e2-bafe-c84044511a7a\") " pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.664804 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.664767 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s55k5\" (UniqueName: \"kubernetes.io/projected/565c20e1-8a81-4655-9155-9de32af9e470-kube-api-access-s55k5\") pod \"aws-ebs-csi-driver-node-mxc7c\" (UID: \"565c20e1-8a81-4655-9155-9de32af9e470\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.665605 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.665565 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw2hb\" (UniqueName: \"kubernetes.io/projected/11d98db9-882d-4480-b299-36d20927e585-kube-api-access-mw2hb\") pod \"multus-additional-cni-plugins-ql2kp\" (UID: \"11d98db9-882d-4480-b299-36d20927e585\") " pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.666195 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.666143 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sblrw\" (UniqueName: \"kubernetes.io/projected/27ee70dd-0218-4c07-a51f-68e69418cbdf-kube-api-access-sblrw\") pod \"ovnkube-node-rhz66\" (UID: \"27ee70dd-0218-4c07-a51f-68e69418cbdf\") " pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.666918 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.666895 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsbkf\" (UniqueName: \"kubernetes.io/projected/390a4dd4-04cc-4db6-93e5-33ee6222c93e-kube-api-access-qsbkf\") pod \"node-ca-84kdj\" (UID: \"390a4dd4-04cc-4db6-93e5-33ee6222c93e\") " pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.667311 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.667268 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn7zl\" (UniqueName: \"kubernetes.io/projected/dfd3af5f-f19f-48a0-97d2-cd472064b05c-kube-api-access-bn7zl\") pod \"iptables-alerter-pt8lv\" (UID: \"dfd3af5f-f19f-48a0-97d2-cd472064b05c\") " pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.742436 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.742403 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bwb2c" Jul 10 12:45:51.758303 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.758271 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pt8lv" Jul 10 12:45:51.766895 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.766879 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:45:51.774400 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.774381 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" Jul 10 12:45:51.781906 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.781886 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6hl8t" Jul 10 12:45:51.789453 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.789435 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:45:51.797053 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.797033 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" Jul 10 12:45:51.803619 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.803602 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-84kdj" Jul 10 12:45:51.810136 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.810119 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" Jul 10 12:45:51.956612 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:51.956533 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 12:45:52.059986 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.059955 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:52.060143 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.060022 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:52.060143 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.060056 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:52.060143 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:52.060108 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 12:45:52.060143 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:52.060130 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 12:45:52.060143 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:52.060140 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kmldp for pod openshift-network-diagnostics/network-check-target-572wk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:52.060366 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:52.060148 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:52.060366 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:52.060148 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:52.060366 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:52.060194 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp podName:eb142d03-253f-4af6-97a8-80adc3e17656 nodeName:}" failed. No retries permitted until 2026-07-10 12:45:53.060181087 +0000 UTC m=+3.945941680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kmldp" (UniqueName: "kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp") pod "network-check-target-572wk" (UID: "eb142d03-253f-4af6-97a8-80adc3e17656") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:52.060366 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:52.060209 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret podName:ea5247d2-85e6-4959-9479-a6eea1212a2d nodeName:}" failed. No retries permitted until 2026-07-10 12:45:53.060202157 +0000 UTC m=+3.945962749 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret") pod "global-pull-secret-syncer-8ln2b" (UID: "ea5247d2-85e6-4959-9479-a6eea1212a2d") : object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:52.060366 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:52.060223 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs podName:daa28724-34bf-4c69-9bec-fcee00bc456f nodeName:}" failed. No retries permitted until 2026-07-10 12:45:53.060215445 +0000 UTC m=+3.945976041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs") pod "network-metrics-daemon-frf69" (UID: "daa28724-34bf-4c69-9bec-fcee00bc456f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:52.471994 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:52.471972 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565c20e1_8a81_4655_9155_9de32af9e470.slice/crio-2e34bd2014c9f5e08b58b34cd6323ca63f480ecd939e8276159cf24b732be651 WatchSource:0}: Error finding container 2e34bd2014c9f5e08b58b34cd6323ca63f480ecd939e8276159cf24b732be651: Status 404 returned error can't find the container with id 2e34bd2014c9f5e08b58b34cd6323ca63f480ecd939e8276159cf24b732be651 Jul 10 12:45:52.473231 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:52.473163 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b3b569_5224_4053_81bb_9ab20a2a397f.slice/crio-dd513d0035663d3affba7f909b8c97010fde3456c1bc043d6e632cf2c8d0f65b WatchSource:0}: Error finding container dd513d0035663d3affba7f909b8c97010fde3456c1bc043d6e632cf2c8d0f65b: Status 404 returned error can't find the container with id dd513d0035663d3affba7f909b8c97010fde3456c1bc043d6e632cf2c8d0f65b Jul 10 12:45:52.474981 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:52.474933 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27ee70dd_0218_4c07_a51f_68e69418cbdf.slice/crio-b34a82169e283f8fc31dacc20ebcdd7718c5f2459283dc06525fec3f655ed8d8 WatchSource:0}: Error finding container b34a82169e283f8fc31dacc20ebcdd7718c5f2459283dc06525fec3f655ed8d8: Status 404 returned error can't find the container with id b34a82169e283f8fc31dacc20ebcdd7718c5f2459283dc06525fec3f655ed8d8 Jul 10 12:45:52.477424 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:52.477365 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfd3af5f_f19f_48a0_97d2_cd472064b05c.slice/crio-062763ac2077fe58f92bd035a56d88527f53f8dc96d2d2f1c929be063531c5c0 WatchSource:0}: Error finding container 062763ac2077fe58f92bd035a56d88527f53f8dc96d2d2f1c929be063531c5c0: Status 404 returned error can't find the container with id 062763ac2077fe58f92bd035a56d88527f53f8dc96d2d2f1c929be063531c5c0 Jul 10 12:45:52.479593 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:52.479565 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d98db9_882d_4480_b299_36d20927e585.slice/crio-2f9ea4f5c256e6ea62429c182d1be38bf024a4aa46dea09f7b5a5f692c60c05a WatchSource:0}: Error finding container 2f9ea4f5c256e6ea62429c182d1be38bf024a4aa46dea09f7b5a5f692c60c05a: Status 404 returned error can't find the container with id 2f9ea4f5c256e6ea62429c182d1be38bf024a4aa46dea09f7b5a5f692c60c05a Jul 10 12:45:52.479896 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:52.479872 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390a4dd4_04cc_4db6_93e5_33ee6222c93e.slice/crio-4c3c1d142864f8f9f6671b38b881eb4588a8be424b6ed2600509d78414c3e511 WatchSource:0}: Error finding container 4c3c1d142864f8f9f6671b38b881eb4588a8be424b6ed2600509d78414c3e511: Status 404 returned error can't find the container with id 4c3c1d142864f8f9f6671b38b881eb4588a8be424b6ed2600509d78414c3e511 Jul 10 12:45:52.500998 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:52.500969 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aeb355e_62c8_4c45_9b4a_795ab74cfe42.slice/crio-b94b854650d644d96fbf1ad9dff3ebe74129ee5c6b97eccb375a0c0ac70b09b8 WatchSource:0}: Error finding container b94b854650d644d96fbf1ad9dff3ebe74129ee5c6b97eccb375a0c0ac70b09b8: Status 404 returned error can't find the container with id b94b854650d644d96fbf1ad9dff3ebe74129ee5c6b97eccb375a0c0ac70b09b8 Jul 10 12:45:52.501827 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:52.501792 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83fab052_d823_4c52_bf03_26323aba154d.slice/crio-5c83d0d1124ce0d20756b1bbf7432b8ef8a4380799c16e92459232148b118c41 WatchSource:0}: Error finding container 5c83d0d1124ce0d20756b1bbf7432b8ef8a4380799c16e92459232148b118c41: Status 404 returned error can't find the container with id 5c83d0d1124ce0d20756b1bbf7432b8ef8a4380799c16e92459232148b118c41 Jul 10 12:45:52.502599 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:45:52.502586 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11de507a_38ab_47e2_bafe_c84044511a7a.slice/crio-f55bd64d1ebfe22893131d7bf36167139c790a2cbb132c86db49d1b5159ce677 WatchSource:0}: Error finding container f55bd64d1ebfe22893131d7bf36167139c790a2cbb132c86db49d1b5159ce677: Status 404 returned error can't find the container with id f55bd64d1ebfe22893131d7bf36167139c790a2cbb132c86db49d1b5159ce677 Jul 10 12:45:52.520529 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.520367 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 12:40:51 +0000 UTC" deadline="2028-04-20 21:47:43.429175386 +0000 UTC" Jul 10 12:45:52.520529 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.520528 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15609h1m50.908650584s" Jul 10 12:45:52.539112 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.539084 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" event={"ID":"11d98db9-882d-4480-b299-36d20927e585","Type":"ContainerStarted","Data":"2f9ea4f5c256e6ea62429c182d1be38bf024a4aa46dea09f7b5a5f692c60c05a"} Jul 10 12:45:52.540214 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.540188 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" event={"ID":"11de507a-38ab-47e2-bafe-c84044511a7a","Type":"ContainerStarted","Data":"f55bd64d1ebfe22893131d7bf36167139c790a2cbb132c86db49d1b5159ce677"} Jul 10 12:45:52.541304 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.541265 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6hl8t" event={"ID":"4aeb355e-62c8-4c45-9b4a-795ab74cfe42","Type":"ContainerStarted","Data":"b94b854650d644d96fbf1ad9dff3ebe74129ee5c6b97eccb375a0c0ac70b09b8"} Jul 10 12:45:52.542232 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.542206 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-84kdj" event={"ID":"390a4dd4-04cc-4db6-93e5-33ee6222c93e","Type":"ContainerStarted","Data":"4c3c1d142864f8f9f6671b38b881eb4588a8be424b6ed2600509d78414c3e511"} Jul 10 12:45:52.543114 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.543090 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pt8lv" event={"ID":"dfd3af5f-f19f-48a0-97d2-cd472064b05c","Type":"ContainerStarted","Data":"062763ac2077fe58f92bd035a56d88527f53f8dc96d2d2f1c929be063531c5c0"} Jul 10 12:45:52.546031 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.546012 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" event={"ID":"27ee70dd-0218-4c07-a51f-68e69418cbdf","Type":"ContainerStarted","Data":"b34a82169e283f8fc31dacc20ebcdd7718c5f2459283dc06525fec3f655ed8d8"} Jul 10 12:45:52.549325 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.549286 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bwb2c" event={"ID":"f2b3b569-5224-4053-81bb-9ab20a2a397f","Type":"ContainerStarted","Data":"dd513d0035663d3affba7f909b8c97010fde3456c1bc043d6e632cf2c8d0f65b"} Jul 10 12:45:52.553235 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.553216 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" event={"ID":"565c20e1-8a81-4655-9155-9de32af9e470","Type":"ContainerStarted","Data":"2e34bd2014c9f5e08b58b34cd6323ca63f480ecd939e8276159cf24b732be651"} Jul 10 12:45:52.554331 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.554311 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gmfsc" event={"ID":"83fab052-d823-4c52-bf03-26323aba154d","Type":"ContainerStarted","Data":"5c83d0d1124ce0d20756b1bbf7432b8ef8a4380799c16e92459232148b118c41"} Jul 10 12:45:52.953590 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:52.953559 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 12:45:53.071383 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.071283 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:53.071383 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.071361 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:53.071594 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.071399 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:53.071594 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.071546 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 12:45:53.071594 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.071564 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 12:45:53.071594 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.071578 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kmldp for pod openshift-network-diagnostics/network-check-target-572wk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:53.071783 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.071635 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp podName:eb142d03-253f-4af6-97a8-80adc3e17656 nodeName:}" failed. No retries permitted until 2026-07-10 12:45:55.071616553 +0000 UTC m=+5.957377173 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kmldp" (UniqueName: "kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp") pod "network-check-target-572wk" (UID: "eb142d03-253f-4af6-97a8-80adc3e17656") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:53.071783 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.071708 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:53.071783 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.071743 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret podName:ea5247d2-85e6-4959-9479-a6eea1212a2d nodeName:}" failed. No retries permitted until 2026-07-10 12:45:55.071732138 +0000 UTC m=+5.957492738 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret") pod "global-pull-secret-syncer-8ln2b" (UID: "ea5247d2-85e6-4959-9479-a6eea1212a2d") : object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:53.071945 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.071796 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:53.071945 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.071828 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs podName:daa28724-34bf-4c69-9bec-fcee00bc456f nodeName:}" failed. No retries permitted until 2026-07-10 12:45:55.0718169 +0000 UTC m=+5.957577493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs") pod "network-metrics-daemon-frf69" (UID: "daa28724-34bf-4c69-9bec-fcee00bc456f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:53.521127 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.521063 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 12:40:51 +0000 UTC" deadline="2027-12-07 15:19:39.10287055 +0000 UTC" Jul 10 12:45:53.521127 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.521097 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12362h33m45.581778325s" Jul 10 12:45:53.535639 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.534556 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:53.535639 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.534680 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:45:53.535639 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.535069 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:53.535639 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.535171 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:45:53.535639 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.535228 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:53.535639 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:53.535307 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:45:53.581980 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.581322 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal" event={"ID":"d1078cbfb33567ae55c66f9d84943893","Type":"ContainerStarted","Data":"3cfd3874b3261144597f1ee7749ba97ea272b3d0cd1a2f40e38538c6641f85ff"} Jul 10 12:45:53.585718 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.585194 2571 generic.go:356] "Generic (PLEG): container finished" podID="6a52decb2df7daf27b74a4f34aa1fb98" containerID="0199ddf4d37664c83224ae8b031b4519b5e9a46f442cca4f8ee8191d05da12d6" exitCode=0 Jul 10 12:45:53.585718 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.585237 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" event={"ID":"6a52decb2df7daf27b74a4f34aa1fb98","Type":"ContainerDied","Data":"0199ddf4d37664c83224ae8b031b4519b5e9a46f442cca4f8ee8191d05da12d6"} Jul 10 12:45:53.618472 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:53.617154 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-162.ec2.internal" podStartSLOduration=3.617131413 podStartE2EDuration="3.617131413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:45:53.601041845 +0000 UTC m=+4.486802461" watchObservedRunningTime="2026-07-10 12:45:53.617131413 +0000 UTC m=+4.502892025" Jul 10 12:45:54.614591 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:54.614447 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" event={"ID":"6a52decb2df7daf27b74a4f34aa1fb98","Type":"ContainerStarted","Data":"ff551741580e3bb5705afbba82f2235a6a667d790b452bbcaf45f8a4582e5a75"} Jul 10 12:45:55.090570 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:55.090481 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:55.090570 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:55.090548 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:55.090804 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:55.090582 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:55.090804 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.090739 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 12:45:55.090804 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.090759 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 12:45:55.090804 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.090773 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kmldp for pod openshift-network-diagnostics/network-check-target-572wk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:55.091001 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.090831 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp podName:eb142d03-253f-4af6-97a8-80adc3e17656 nodeName:}" failed. No retries permitted until 2026-07-10 12:45:59.090812242 +0000 UTC m=+9.976572840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kmldp" (UniqueName: "kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp") pod "network-check-target-572wk" (UID: "eb142d03-253f-4af6-97a8-80adc3e17656") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:55.091001 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.090905 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:55.091001 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.090938 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret podName:ea5247d2-85e6-4959-9479-a6eea1212a2d nodeName:}" failed. No retries permitted until 2026-07-10 12:45:59.090927217 +0000 UTC m=+9.976687815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret") pod "global-pull-secret-syncer-8ln2b" (UID: "ea5247d2-85e6-4959-9479-a6eea1212a2d") : object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:55.091001 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.090987 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:55.091206 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.091015 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs podName:daa28724-34bf-4c69-9bec-fcee00bc456f nodeName:}" failed. No retries permitted until 2026-07-10 12:45:59.091003138 +0000 UTC m=+9.976763730 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs") pod "network-metrics-daemon-frf69" (UID: "daa28724-34bf-4c69-9bec-fcee00bc456f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:55.531103 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:55.530394 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:55.531103 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.530523 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:45:55.531103 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:55.530562 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:55.531103 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.530639 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:45:55.531103 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:55.530693 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:55.531103 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:55.530753 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:45:57.529840 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:57.529253 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:57.529840 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:57.529272 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:57.529840 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:57.529409 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:45:57.529840 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:57.529510 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:45:57.529840 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:57.529568 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:57.529840 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:57.529640 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:45:59.123446 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:59.123395 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:59.123446 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:59.123444 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:59.123956 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:59.123501 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:59.123956 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.123614 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:59.123956 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.123671 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret podName:ea5247d2-85e6-4959-9479-a6eea1212a2d nodeName:}" failed. No retries permitted until 2026-07-10 12:46:07.12365249 +0000 UTC m=+18.009413087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret") pod "global-pull-secret-syncer-8ln2b" (UID: "ea5247d2-85e6-4959-9479-a6eea1212a2d") : object "kube-system"/"original-pull-secret" not registered Jul 10 12:45:59.123956 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.123739 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:59.123956 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.123775 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs podName:daa28724-34bf-4c69-9bec-fcee00bc456f nodeName:}" failed. No retries permitted until 2026-07-10 12:46:07.123761885 +0000 UTC m=+18.009522482 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs") pod "network-metrics-daemon-frf69" (UID: "daa28724-34bf-4c69-9bec-fcee00bc456f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:45:59.123956 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.123850 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 12:45:59.123956 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.123867 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 12:45:59.123956 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.123879 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kmldp for pod openshift-network-diagnostics/network-check-target-572wk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:59.123956 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.123910 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp podName:eb142d03-253f-4af6-97a8-80adc3e17656 nodeName:}" failed. No retries permitted until 2026-07-10 12:46:07.123900639 +0000 UTC m=+18.009661235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kmldp" (UniqueName: "kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp") pod "network-check-target-572wk" (UID: "eb142d03-253f-4af6-97a8-80adc3e17656") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:45:59.533648 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:59.530898 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:45:59.533648 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.531033 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:45:59.533648 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:59.531466 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:45:59.533648 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.531596 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:45:59.533648 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:45:59.531645 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:45:59.533648 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:45:59.531732 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:01.529746 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:01.529714 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:01.530198 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:01.529714 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:01.530198 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:01.529714 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:01.530198 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:01.529972 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:01.530198 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:01.529846 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:01.530198 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:01.530070 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:03.529054 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:03.529015 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:03.529054 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:03.529044 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:03.529561 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:03.529015 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:03.529561 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:03.529149 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:03.529561 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:03.529236 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:03.529561 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:03.529330 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:05.529860 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:05.529823 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:05.530307 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:05.529823 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:05.530307 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:05.529939 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:05.530307 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:05.529960 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:05.530307 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:05.530062 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:05.530307 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:05.530143 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:07.184303 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:07.184266 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:07.184323 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:07.184359 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.184439 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.184483 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret podName:ea5247d2-85e6-4959-9479-a6eea1212a2d nodeName:}" failed. No retries permitted until 2026-07-10 12:46:23.184469801 +0000 UTC m=+34.070230395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret") pod "global-pull-secret-syncer-8ln2b" (UID: "ea5247d2-85e6-4959-9479-a6eea1212a2d") : object "kube-system"/"original-pull-secret" not registered Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.184484 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.184504 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.184513 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kmldp for pod openshift-network-diagnostics/network-check-target-572wk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.184437 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.184549 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp podName:eb142d03-253f-4af6-97a8-80adc3e17656 nodeName:}" failed. No retries permitted until 2026-07-10 12:46:23.18453879 +0000 UTC m=+34.070299403 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kmldp" (UniqueName: "kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp") pod "network-check-target-572wk" (UID: "eb142d03-253f-4af6-97a8-80adc3e17656") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:46:07.184755 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.184573 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs podName:daa28724-34bf-4c69-9bec-fcee00bc456f nodeName:}" failed. No retries permitted until 2026-07-10 12:46:23.184555487 +0000 UTC m=+34.070316085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs") pod "network-metrics-daemon-frf69" (UID: "daa28724-34bf-4c69-9bec-fcee00bc456f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:46:07.529096 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:07.529014 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:07.529242 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:07.529025 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:07.529242 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:07.529142 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:07.529242 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.529142 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:07.529409 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.529241 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:07.529409 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:07.529330 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:09.530005 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:09.529974 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:09.530451 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:09.530063 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:09.530451 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:09.530072 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:09.530451 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:09.530187 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:09.530451 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:09.530222 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:09.530451 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:09.530332 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:11.529026 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.528848 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:11.529637 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.528856 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:11.529637 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:11.529126 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:11.529637 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:11.529230 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:11.529637 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.528905 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:11.529637 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:11.529340 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:11.650252 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.650220 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gmfsc" event={"ID":"83fab052-d823-4c52-bf03-26323aba154d","Type":"ContainerStarted","Data":"fb7166fcac94766c00f4ee26e004ceaaf26e55fe8fff0becb36e65fabc656a7a"} Jul 10 12:46:11.651932 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.651907 2571 generic.go:356] "Generic (PLEG): container finished" podID="11d98db9-882d-4480-b299-36d20927e585" containerID="bb34efb39b68efaa408ca0c0bb186c685fe667e2cfa139dd9880bc09876f480c" exitCode=0 Jul 10 12:46:11.652048 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.651990 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" event={"ID":"11d98db9-882d-4480-b299-36d20927e585","Type":"ContainerDied","Data":"bb34efb39b68efaa408ca0c0bb186c685fe667e2cfa139dd9880bc09876f480c"} Jul 10 12:46:11.653811 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.653764 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" event={"ID":"11de507a-38ab-47e2-bafe-c84044511a7a","Type":"ContainerStarted","Data":"153e9589a39a6bc2c4e7e36ade16049631eb018ab750cd8bfefd2acc3b77591c"} Jul 10 12:46:11.656494 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.656474 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6hl8t" event={"ID":"4aeb355e-62c8-4c45-9b4a-795ab74cfe42","Type":"ContainerStarted","Data":"029ff831ad0768e86305efae66aff5af13f72b7af62ab651318b346565824625"} Jul 10 12:46:11.658553 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.658525 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-84kdj" event={"ID":"390a4dd4-04cc-4db6-93e5-33ee6222c93e","Type":"ContainerStarted","Data":"84518e0b4afb46cd383b4671188cf658dc82fbbddfb9690ee67b9f5d0ca2ef5d"} Jul 10 12:46:11.661820 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.661796 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" event={"ID":"27ee70dd-0218-4c07-a51f-68e69418cbdf","Type":"ContainerStarted","Data":"469719d2328fe46eaee85f3c5e779a52eeb700f95ae23c03ea9898f14c75ba5b"} Jul 10 12:46:11.661909 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.661872 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" event={"ID":"27ee70dd-0218-4c07-a51f-68e69418cbdf","Type":"ContainerStarted","Data":"f46508a1f2664db8a84fcc025a2ec95acc22072baea3ae01de75acba72bffabf"} Jul 10 12:46:11.661954 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.661926 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" event={"ID":"27ee70dd-0218-4c07-a51f-68e69418cbdf","Type":"ContainerStarted","Data":"ec923bcec8f3c8cdb1bbfd3be1ddf3f00392917b789cfdeb68ef8ca72694d792"} Jul 10 12:46:11.662004 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.661977 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" event={"ID":"27ee70dd-0218-4c07-a51f-68e69418cbdf","Type":"ContainerStarted","Data":"d320c2b4f3c50c6fc5ad40cb3d77ad84914df25ca533db7d5a6ee6cd14fd685d"} Jul 10 12:46:11.662061 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.662023 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" event={"ID":"27ee70dd-0218-4c07-a51f-68e69418cbdf","Type":"ContainerStarted","Data":"80fc391f4a91d08c2510f44f20ac3a39811eb2ee0c7d424ae32c206af9c23994"} Jul 10 12:46:11.662113 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.662070 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" event={"ID":"27ee70dd-0218-4c07-a51f-68e69418cbdf","Type":"ContainerStarted","Data":"aefd216f9bad756a96221f9e25745f53cad92a536f0bc9063cf07247ed91ce44"} Jul 10 12:46:11.663551 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.663528 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bwb2c" event={"ID":"f2b3b569-5224-4053-81bb-9ab20a2a397f","Type":"ContainerStarted","Data":"ad270f556c4ff483b873b94308398d11697cd48500413781313cdd41c2001b96"} Jul 10 12:46:11.665133 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.665105 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" event={"ID":"565c20e1-8a81-4655-9155-9de32af9e470","Type":"ContainerStarted","Data":"5ec336e7788ec13453d99369544021de09c1791d51094399836c9bb813ccedf3"} Jul 10 12:46:11.672220 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.672156 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gmfsc" podStartSLOduration=12.158470029 podStartE2EDuration="22.672137735s" totalImagesPullingTime="10.513667706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:11.67148269 +0000 UTC m=+22.557243306" watchObservedRunningTime="2026-07-10 12:46:11.672137735 +0000 UTC m=+22.557898352" Jul 10 12:46:11.672436 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.672396 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-162.ec2.internal" podStartSLOduration=21.672387593 podStartE2EDuration="21.672387593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:45:54.631273101 +0000 UTC m=+5.517033756" watchObservedRunningTime="2026-07-10 12:46:11.672387593 +0000 UTC m=+22.558148210" Jul 10 12:46:11.691031 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.690988 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tmnx8" podStartSLOduration=4.303861103 podStartE2EDuration="22.690973077s" totalImagesPullingTime="18.387111974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:11.690482681 +0000 UTC m=+22.576243297" watchObservedRunningTime="2026-07-10 12:46:11.690973077 +0000 UTC m=+22.576733691" Jul 10 12:46:11.749047 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.748964 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bwb2c" podStartSLOduration=4.279922891 podStartE2EDuration="22.748951446s" totalImagesPullingTime="18.469028555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:11.748565494 +0000 UTC m=+22.634326111" watchObservedRunningTime="2026-07-10 12:46:11.748951446 +0000 UTC m=+22.634712105" Jul 10 12:46:11.772507 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.772468 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6hl8t" podStartSLOduration=4.38783973 podStartE2EDuration="22.772456598s" totalImagesPullingTime="18.384616868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:11.772440894 +0000 UTC m=+22.658201510" watchObservedRunningTime="2026-07-10 12:46:11.772456598 +0000 UTC m=+22.658217213" Jul 10 12:46:11.787994 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:11.787954 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-84kdj" podStartSLOduration=4.399485298 podStartE2EDuration="22.787941421s" totalImagesPullingTime="18.388456123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:11.787873196 +0000 UTC m=+22.673633812" watchObservedRunningTime="2026-07-10 12:46:11.787941421 +0000 UTC m=+22.673702035" Jul 10 12:46:12.370783 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:12.370758 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 12:46:12.404324 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:12.404284 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:46:12.404886 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:12.404859 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:46:12.504650 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:12.504570 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T12:46:12.370780348Z","UUID":"0c593586-62a8-4f67-808b-a0e17c937fd6","Handler":null,"Name":"","Endpoint":""} Jul 10 12:46:12.506197 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:12.506166 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 12:46:12.506338 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:12.506206 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 12:46:12.669051 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:12.668975 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pt8lv" event={"ID":"dfd3af5f-f19f-48a0-97d2-cd472064b05c","Type":"ContainerStarted","Data":"fad569066a1f451af3d6b224bd94fb635e63fa2e0f110601e750b584aad8ce3d"} Jul 10 12:46:12.670963 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:12.670933 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" event={"ID":"565c20e1-8a81-4655-9155-9de32af9e470","Type":"ContainerStarted","Data":"7c3becc0b4af19e9ba6bf93b1279abf8d525504953a6023d703a4d9354b16d7e"} Jul 10 12:46:12.684423 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:12.684380 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pt8lv" podStartSLOduration=5.294671273 podStartE2EDuration="23.684362961s" totalImagesPullingTime="18.389691688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:12.68407851 +0000 UTC m=+23.569839126" watchObservedRunningTime="2026-07-10 12:46:12.684362961 +0000 UTC m=+23.570123575" Jul 10 12:46:13.532449 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:13.532420 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:13.532449 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:13.532436 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:13.532449 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:13.532445 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:13.532758 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:13.532535 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:13.532758 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:13.532628 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:13.532758 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:13.532729 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:13.676173 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:13.676150 2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 12:46:13.676173 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:13.676141 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" event={"ID":"565c20e1-8a81-4655-9155-9de32af9e470","Type":"ContainerStarted","Data":"754725c4084373d44c0bad31b8c03976b62ce96e24a8a30fab1e2d6156b8fcdd"} Jul 10 12:46:13.697863 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:13.697823 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mxc7c" podStartSLOduration=4.147675551 podStartE2EDuration="24.69780571s" totalImagesPullingTime="20.550130159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:13.697775544 +0000 UTC m=+24.583536158" watchObservedRunningTime="2026-07-10 12:46:13.69780571 +0000 UTC m=+24.583566321" Jul 10 12:46:14.682188 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:14.681951 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" event={"ID":"27ee70dd-0218-4c07-a51f-68e69418cbdf","Type":"ContainerStarted","Data":"776067130e7eeb76020f4dbe6375a9531717ae31ab78e4f23ebba7526bff6078"} Jul 10 12:46:15.529106 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:15.529075 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:15.529314 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:15.529077 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:15.529314 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:15.529196 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:15.529314 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:15.529252 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:15.529314 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:15.529077 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:15.529493 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:15.529346 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:16.689618 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:16.689404 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" event={"ID":"27ee70dd-0218-4c07-a51f-68e69418cbdf","Type":"ContainerStarted","Data":"e5f354cf2ff4cea70a170924b4a0b46f91a5552898cfe2b6f3fbda89fb07d62d"} Jul 10 12:46:16.690281 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:16.689746 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:46:16.691175 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:16.691150 2571 generic.go:356] "Generic (PLEG): container finished" podID="11d98db9-882d-4480-b299-36d20927e585" containerID="e9846bd7057235b16cfee1505b7b9dc10d7311cf4cd63233876b8dedbc32f62c" exitCode=0 Jul 10 12:46:16.691285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:16.691191 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" event={"ID":"11d98db9-882d-4480-b299-36d20927e585","Type":"ContainerDied","Data":"e9846bd7057235b16cfee1505b7b9dc10d7311cf4cd63233876b8dedbc32f62c"} Jul 10 12:46:16.705916 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:16.705897 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:46:16.754362 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:16.754326 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" podStartSLOduration=9.275070813 podStartE2EDuration="27.754313629s" totalImagesPullingTime="18.479242816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:16.724443747 +0000 UTC m=+27.610204361" watchObservedRunningTime="2026-07-10 12:46:16.754313629 +0000 UTC m=+27.640074284" Jul 10 12:46:17.530780 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:17.530624 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:17.530780 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:17.530640 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:17.530780 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:17.530751 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:17.531086 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:17.530825 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:17.531086 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:17.530889 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:17.531086 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:17.530968 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:17.693861 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:17.693835 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:46:17.694337 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:17.693913 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:46:17.708404 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:17.708284 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:46:18.564092 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:18.564017 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8ln2b"] Jul 10 12:46:18.564415 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:18.564131 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:18.564415 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:18.564231 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:18.565222 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:18.565201 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-572wk"] Jul 10 12:46:18.565321 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:18.565277 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:18.565400 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:18.565375 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:18.568214 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:18.568195 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-frf69"] Jul 10 12:46:18.568308 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:18.568273 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:18.568391 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:18.568370 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:18.697752 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:18.697723 2571 generic.go:356] "Generic (PLEG): container finished" podID="11d98db9-882d-4480-b299-36d20927e585" containerID="e2ae7652703dffc076258d2503da7fad789c9b021b78d0bea3879cec0710dd41" exitCode=0 Jul 10 12:46:18.698098 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:18.697809 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" event={"ID":"11d98db9-882d-4480-b299-36d20927e585","Type":"ContainerDied","Data":"e2ae7652703dffc076258d2503da7fad789c9b021b78d0bea3879cec0710dd41"} Jul 10 12:46:20.529831 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:20.529697 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:20.530142 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:20.529759 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:20.530142 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:20.529901 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:20.530142 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:20.529758 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:20.530142 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:20.529981 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:20.530142 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:20.530106 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:20.703827 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:20.703749 2571 generic.go:356] "Generic (PLEG): container finished" podID="11d98db9-882d-4480-b299-36d20927e585" containerID="9db58a1e97b918b0168715de6313083138770e72c8fa5c17223be2c738881a36" exitCode=0 Jul 10 12:46:20.703827 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:20.703790 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" event={"ID":"11d98db9-882d-4480-b299-36d20927e585","Type":"ContainerDied","Data":"9db58a1e97b918b0168715de6313083138770e72c8fa5c17223be2c738881a36"} Jul 10 12:46:21.174073 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:21.174047 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:46:21.174237 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:21.174158 2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 12:46:21.174727 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:21.174710 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gmfsc" Jul 10 12:46:22.529594 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:22.529557 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:22.530128 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:22.529604 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:22.530128 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:22.529604 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:22.530128 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:22.529684 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-572wk" podUID="eb142d03-253f-4af6-97a8-80adc3e17656" Jul 10 12:46:22.530128 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:22.529793 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-frf69" podUID="daa28724-34bf-4c69-9bec-fcee00bc456f" Jul 10 12:46:22.530128 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:22.529884 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8ln2b" podUID="ea5247d2-85e6-4959-9479-a6eea1212a2d" Jul 10 12:46:23.207597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.207568 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:23.207765 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.207603 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:23.207765 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.207627 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:23.207765 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.207704 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 12:46:23.207765 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.207720 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:46:23.207765 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.207735 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 12:46:23.207765 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.207751 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 12:46:23.207765 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.207763 2571 projected.go:196] Error preparing data for projected volume kube-api-access-kmldp for pod openshift-network-diagnostics/network-check-target-572wk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:46:23.207997 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.207764 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret podName:ea5247d2-85e6-4959-9479-a6eea1212a2d nodeName:}" failed. No retries permitted until 2026-07-10 12:46:55.207748582 +0000 UTC m=+66.093509175 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret") pod "global-pull-secret-syncer-8ln2b" (UID: "ea5247d2-85e6-4959-9479-a6eea1212a2d") : object "kube-system"/"original-pull-secret" not registered Jul 10 12:46:23.207997 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.207801 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs podName:daa28724-34bf-4c69-9bec-fcee00bc456f nodeName:}" failed. No retries permitted until 2026-07-10 12:46:55.207789914 +0000 UTC m=+66.093550507 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs") pod "network-metrics-daemon-frf69" (UID: "daa28724-34bf-4c69-9bec-fcee00bc456f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 12:46:23.207997 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.207813 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp podName:eb142d03-253f-4af6-97a8-80adc3e17656 nodeName:}" failed. No retries permitted until 2026-07-10 12:46:55.207806154 +0000 UTC m=+66.093566747 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kmldp" (UniqueName: "kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp") pod "network-check-target-572wk" (UID: "eb142d03-253f-4af6-97a8-80adc3e17656") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 12:46:23.477501 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.477439 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-162.ec2.internal" event="NodeReady" Jul 10 12:46:23.477616 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.477563 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 12:46:23.509586 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.509560 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dbb8b4668-6bfdw"] Jul 10 12:46:23.511413 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.511394 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.513844 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.513824 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 12:46:23.513943 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.513866 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 12:46:23.514008 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.513983 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ch6zw\"" Jul 10 12:46:23.514104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.514088 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 12:46:23.520762 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.520746 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 12:46:23.524970 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.524952 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p7l2z"] Jul 10 12:46:23.526624 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.526607 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dbb8b4668-6bfdw"] Jul 10 12:46:23.526716 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.526703 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.529203 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.529159 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 12:46:23.529358 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.529208 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 12:46:23.529358 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.529235 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qfssw\"" Jul 10 12:46:23.532485 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.532436 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-prpw8"] Jul 10 12:46:23.536524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.536506 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prpw8" Jul 10 12:46:23.539267 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.539249 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 12:46:23.539430 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.539407 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 12:46:23.539545 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.539437 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8vh7j\"" Jul 10 12:46:23.539545 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.539274 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 12:46:23.539655 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.539344 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p7l2z"] Jul 10 12:46:23.541528 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.541508 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prpw8"] Jul 10 12:46:23.609814 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.609789 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-trusted-ca\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.609965 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.609840 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-certificates\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.609965 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.609871 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.609965 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.609898 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-ca-trust-extracted\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.609965 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.609919 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88jnn\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-kube-api-access-88jnn\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.610141 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.610041 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-image-registry-private-configuration\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.610141 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.610082 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-installation-pull-secrets\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.610141 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.610109 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-bound-sa-token\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.710379 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710350 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0694c98e-3ce3-45cb-8503-47b1acd27007-tmp-dir\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.710560 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710400 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-trusted-ca\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.710560 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710460 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-certificates\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.710560 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710488 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.710560 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710513 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.710789 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710601 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-ca-trust-extracted\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.710789 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710617 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88jnn\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-kube-api-access-88jnn\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.710789 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710637 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjp52\" (UniqueName: \"kubernetes.io/projected/4f2f42e7-8190-41ef-a9a6-ec8b351dc05e-kube-api-access-wjp52\") pod \"ingress-canary-prpw8\" (UID: \"4f2f42e7-8190-41ef-a9a6-ec8b351dc05e\") " pod="openshift-ingress-canary/ingress-canary-prpw8" Jul 10 12:46:23.710789 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710677 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f2f42e7-8190-41ef-a9a6-ec8b351dc05e-cert\") pod \"ingress-canary-prpw8\" (UID: \"4f2f42e7-8190-41ef-a9a6-ec8b351dc05e\") " pod="openshift-ingress-canary/ingress-canary-prpw8" Jul 10 12:46:23.710789 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.710691 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 12:46:23.710789 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.710706 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dbb8b4668-6bfdw: secret "image-registry-tls" not found Jul 10 12:46:23.710789 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710718 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0694c98e-3ce3-45cb-8503-47b1acd27007-config-volume\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.710789 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710747 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-image-registry-private-configuration\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.710789 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.710770 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls podName:4aec912a-9c70-489e-b0f2-1dce7f37e7ee nodeName:}" failed. No retries permitted until 2026-07-10 12:46:24.210750263 +0000 UTC m=+35.096510860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls") pod "image-registry-6dbb8b4668-6bfdw" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee") : secret "image-registry-tls" not found Jul 10 12:46:23.711229 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710890 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-installation-pull-secrets\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.711229 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710930 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-bound-sa-token\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.711229 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.710959 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc4fb\" (UniqueName: \"kubernetes.io/projected/0694c98e-3ce3-45cb-8503-47b1acd27007-kube-api-access-wc4fb\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.711229 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.711004 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-ca-trust-extracted\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.711229 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.711130 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-certificates\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.711503 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.711459 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-trusted-ca\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.714966 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.714939 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-installation-pull-secrets\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.715060 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.714940 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-image-registry-private-configuration\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.734672 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.734622 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-bound-sa-token\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.741472 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.741446 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88jnn\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-kube-api-access-88jnn\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:23.812339 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.812283 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc4fb\" (UniqueName: \"kubernetes.io/projected/0694c98e-3ce3-45cb-8503-47b1acd27007-kube-api-access-wc4fb\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.812550 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.812364 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0694c98e-3ce3-45cb-8503-47b1acd27007-tmp-dir\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.812550 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.812437 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.812550 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.812481 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjp52\" (UniqueName: \"kubernetes.io/projected/4f2f42e7-8190-41ef-a9a6-ec8b351dc05e-kube-api-access-wjp52\") pod \"ingress-canary-prpw8\" (UID: \"4f2f42e7-8190-41ef-a9a6-ec8b351dc05e\") " pod="openshift-ingress-canary/ingress-canary-prpw8" Jul 10 12:46:23.812550 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.812518 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f2f42e7-8190-41ef-a9a6-ec8b351dc05e-cert\") pod \"ingress-canary-prpw8\" (UID: \"4f2f42e7-8190-41ef-a9a6-ec8b351dc05e\") " pod="openshift-ingress-canary/ingress-canary-prpw8" Jul 10 12:46:23.812759 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.812554 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0694c98e-3ce3-45cb-8503-47b1acd27007-config-volume\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.812759 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.812584 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 12:46:23.812759 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:23.812658 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls podName:0694c98e-3ce3-45cb-8503-47b1acd27007 nodeName:}" failed. No retries permitted until 2026-07-10 12:46:24.312638466 +0000 UTC m=+35.198399078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls") pod "dns-default-p7l2z" (UID: "0694c98e-3ce3-45cb-8503-47b1acd27007") : secret "dns-default-metrics-tls" not found Jul 10 12:46:23.812759 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.812731 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0694c98e-3ce3-45cb-8503-47b1acd27007-tmp-dir\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.813086 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.813059 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0694c98e-3ce3-45cb-8503-47b1acd27007-config-volume\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.815356 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.815333 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f2f42e7-8190-41ef-a9a6-ec8b351dc05e-cert\") pod \"ingress-canary-prpw8\" (UID: \"4f2f42e7-8190-41ef-a9a6-ec8b351dc05e\") " pod="openshift-ingress-canary/ingress-canary-prpw8" Jul 10 12:46:23.821212 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.821189 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc4fb\" (UniqueName: \"kubernetes.io/projected/0694c98e-3ce3-45cb-8503-47b1acd27007-kube-api-access-wc4fb\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:23.821467 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.821451 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjp52\" (UniqueName: \"kubernetes.io/projected/4f2f42e7-8190-41ef-a9a6-ec8b351dc05e-kube-api-access-wjp52\") pod \"ingress-canary-prpw8\" (UID: \"4f2f42e7-8190-41ef-a9a6-ec8b351dc05e\") " pod="openshift-ingress-canary/ingress-canary-prpw8" Jul 10 12:46:23.845992 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.845975 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prpw8" Jul 10 12:46:23.992999 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:23.992699 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prpw8"] Jul 10 12:46:24.216402 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.216369 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:24.216564 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:24.216538 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 12:46:24.216564 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:24.216560 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dbb8b4668-6bfdw: secret "image-registry-tls" not found Jul 10 12:46:24.216644 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:24.216615 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls podName:4aec912a-9c70-489e-b0f2-1dce7f37e7ee nodeName:}" failed. No retries permitted until 2026-07-10 12:46:25.216599639 +0000 UTC m=+36.102360232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls") pod "image-registry-6dbb8b4668-6bfdw" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee") : secret "image-registry-tls" not found Jul 10 12:46:24.317495 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.317414 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:24.317650 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:24.317574 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 12:46:24.317706 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:24.317652 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls podName:0694c98e-3ce3-45cb-8503-47b1acd27007 nodeName:}" failed. No retries permitted until 2026-07-10 12:46:25.317631706 +0000 UTC m=+36.203392305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls") pod "dns-default-p7l2z" (UID: "0694c98e-3ce3-45cb-8503-47b1acd27007") : secret "dns-default-metrics-tls" not found Jul 10 12:46:24.529787 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.529751 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:24.529953 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.529751 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:24.529953 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.529758 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:24.533778 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.533757 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 12:46:24.534239 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.533808 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 12:46:24.534239 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.534032 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 12:46:24.534239 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.534070 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-99bwq\"" Jul 10 12:46:24.534239 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.534109 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 12:46:24.534239 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:24.534220 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pmvx9\"" Jul 10 12:46:25.224203 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:25.224165 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:25.224427 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:25.224350 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 12:46:25.224427 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:25.224371 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dbb8b4668-6bfdw: secret "image-registry-tls" not found Jul 10 12:46:25.224539 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:25.224433 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls podName:4aec912a-9c70-489e-b0f2-1dce7f37e7ee nodeName:}" failed. No retries permitted until 2026-07-10 12:46:27.224415376 +0000 UTC m=+38.110175969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls") pod "image-registry-6dbb8b4668-6bfdw" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee") : secret "image-registry-tls" not found Jul 10 12:46:25.325125 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:25.325091 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:25.325320 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:25.325263 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 12:46:25.325420 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:25.325334 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls podName:0694c98e-3ce3-45cb-8503-47b1acd27007 nodeName:}" failed. No retries permitted until 2026-07-10 12:46:27.325320695 +0000 UTC m=+38.211081288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls") pod "dns-default-p7l2z" (UID: "0694c98e-3ce3-45cb-8503-47b1acd27007") : secret "dns-default-metrics-tls" not found Jul 10 12:46:25.398542 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:46:25.398512 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2f42e7_8190_41ef_a9a6_ec8b351dc05e.slice/crio-01d48b5020271bb79c616d7a8ca66ff6477ec0fcb479e0eaf71f9c15bd6524cf WatchSource:0}: Error finding container 01d48b5020271bb79c616d7a8ca66ff6477ec0fcb479e0eaf71f9c15bd6524cf: Status 404 returned error can't find the container with id 01d48b5020271bb79c616d7a8ca66ff6477ec0fcb479e0eaf71f9c15bd6524cf Jul 10 12:46:25.717666 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:25.717616 2571 generic.go:356] "Generic (PLEG): container finished" podID="11d98db9-882d-4480-b299-36d20927e585" containerID="49c30914603ca061a415ee8550e4f9eb4178c6a72d5792ed1f41ca084d443060" exitCode=0 Jul 10 12:46:25.718028 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:25.717710 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" event={"ID":"11d98db9-882d-4480-b299-36d20927e585","Type":"ContainerDied","Data":"49c30914603ca061a415ee8550e4f9eb4178c6a72d5792ed1f41ca084d443060"} Jul 10 12:46:25.718796 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:25.718776 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prpw8" event={"ID":"4f2f42e7-8190-41ef-a9a6-ec8b351dc05e","Type":"ContainerStarted","Data":"01d48b5020271bb79c616d7a8ca66ff6477ec0fcb479e0eaf71f9c15bd6524cf"} Jul 10 12:46:26.725553 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:26.725510 2571 generic.go:356] "Generic (PLEG): container finished" podID="11d98db9-882d-4480-b299-36d20927e585" containerID="5ba2f49a1c070d555df9a3d57fd96a1dc52c68d802fd0b204a74f10f2785c157" exitCode=0 Jul 10 12:46:26.726053 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:26.725563 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" event={"ID":"11d98db9-882d-4480-b299-36d20927e585","Type":"ContainerDied","Data":"5ba2f49a1c070d555df9a3d57fd96a1dc52c68d802fd0b204a74f10f2785c157"} Jul 10 12:46:27.239071 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:27.239035 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:27.239229 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:27.239195 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 12:46:27.239229 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:27.239214 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dbb8b4668-6bfdw: secret "image-registry-tls" not found Jul 10 12:46:27.239332 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:27.239268 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls podName:4aec912a-9c70-489e-b0f2-1dce7f37e7ee nodeName:}" failed. No retries permitted until 2026-07-10 12:46:31.239253893 +0000 UTC m=+42.125014486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls") pod "image-registry-6dbb8b4668-6bfdw" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee") : secret "image-registry-tls" not found Jul 10 12:46:27.340307 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:27.340263 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:27.340472 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:27.340414 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 12:46:27.340523 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:27.340488 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls podName:0694c98e-3ce3-45cb-8503-47b1acd27007 nodeName:}" failed. No retries permitted until 2026-07-10 12:46:31.340467526 +0000 UTC m=+42.226228120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls") pod "dns-default-p7l2z" (UID: "0694c98e-3ce3-45cb-8503-47b1acd27007") : secret "dns-default-metrics-tls" not found Jul 10 12:46:27.732517 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:27.732480 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" event={"ID":"11d98db9-882d-4480-b299-36d20927e585","Type":"ContainerStarted","Data":"3a6fcb2c4dfaa4f741da09a40921c30105570be2ec4d0275cf9ebc8c704daff4"} Jul 10 12:46:27.757816 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:27.757558 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ql2kp" podStartSLOduration=8.275707111 podStartE2EDuration="38.757536519s" totalImagesPullingTime="30.481829408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:27.756166546 +0000 UTC m=+38.641927161" watchObservedRunningTime="2026-07-10 12:46:27.757536519 +0000 UTC m=+38.643297135" Jul 10 12:46:28.736100 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:28.736057 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prpw8" event={"ID":"4f2f42e7-8190-41ef-a9a6-ec8b351dc05e","Type":"ContainerStarted","Data":"ef57f3ffcf9e2bcb699ae8fe61b140fa9260d0abd0fadf7431c1dc4edc4773ea"} Jul 10 12:46:28.760721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:28.760675 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-prpw8" podStartSLOduration=3.122963876 podStartE2EDuration="5.760661465s" totalImagesPullingTime="2.637697589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:46:28.759402514 +0000 UTC m=+39.645163129" watchObservedRunningTime="2026-07-10 12:46:28.760661465 +0000 UTC m=+39.646422073" Jul 10 12:46:31.267031 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:31.266994 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:31.267390 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:31.267143 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 12:46:31.267390 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:31.267161 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dbb8b4668-6bfdw: secret "image-registry-tls" not found Jul 10 12:46:31.267390 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:31.267219 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls podName:4aec912a-9c70-489e-b0f2-1dce7f37e7ee nodeName:}" failed. No retries permitted until 2026-07-10 12:46:39.267201393 +0000 UTC m=+50.152961989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls") pod "image-registry-6dbb8b4668-6bfdw" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee") : secret "image-registry-tls" not found Jul 10 12:46:31.367310 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:31.367263 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:31.367438 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:31.367411 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 12:46:31.367481 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:31.367467 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls podName:0694c98e-3ce3-45cb-8503-47b1acd27007 nodeName:}" failed. No retries permitted until 2026-07-10 12:46:39.367451623 +0000 UTC m=+50.253212219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls") pod "dns-default-p7l2z" (UID: "0694c98e-3ce3-45cb-8503-47b1acd27007") : secret "dns-default-metrics-tls" not found Jul 10 12:46:39.317347 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:39.317305 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:39.317831 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:39.317412 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 12:46:39.317831 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:39.317423 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dbb8b4668-6bfdw: secret "image-registry-tls" not found Jul 10 12:46:39.317831 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:39.317473 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls podName:4aec912a-9c70-489e-b0f2-1dce7f37e7ee nodeName:}" failed. No retries permitted until 2026-07-10 12:46:55.317458777 +0000 UTC m=+66.203219370 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls") pod "image-registry-6dbb8b4668-6bfdw" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee") : secret "image-registry-tls" not found Jul 10 12:46:39.417757 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:39.417719 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:39.417892 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:39.417852 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 12:46:39.417940 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:39.417916 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls podName:0694c98e-3ce3-45cb-8503-47b1acd27007 nodeName:}" failed. No retries permitted until 2026-07-10 12:46:55.417901454 +0000 UTC m=+66.303662048 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls") pod "dns-default-p7l2z" (UID: "0694c98e-3ce3-45cb-8503-47b1acd27007") : secret "dns-default-metrics-tls" not found Jul 10 12:46:49.718410 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:49.718383 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rhz66" Jul 10 12:46:55.149091 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.149058 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-794657f478-2bsbk"] Jul 10 12:46:55.151366 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.151349 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.153891 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.153864 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gxg5l\"" Jul 10 12:46:55.154018 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.153906 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 12:46:55.154018 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.153925 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 12:46:55.154018 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.153932 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 12:46:55.154018 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.153931 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 12:46:55.154191 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.154024 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 12:46:55.154326 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.154312 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 12:46:55.164803 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.164784 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-794657f478-2bsbk"] Jul 10 12:46:55.220029 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.220010 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.220029 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.220036 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.220186 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.220065 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:55.220186 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.220114 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-stats-auth\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.220186 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.220180 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zggg\" (UniqueName: \"kubernetes.io/projected/77b92f38-8541-4c6c-b440-e05f745570ec-kube-api-access-4zggg\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.220317 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.220209 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:46:55.220317 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.220262 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:55.220390 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.220310 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-default-certificate\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.222725 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.222707 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 12:46:55.222816 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.222802 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 12:46:55.225856 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.223183 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 12:46:55.230378 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.230363 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 12:46:55.230460 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.230435 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs podName:daa28724-34bf-4c69-9bec-fcee00bc456f nodeName:}" failed. No retries permitted until 2026-07-10 12:47:59.230413856 +0000 UTC m=+130.116174463 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs") pod "network-metrics-daemon-frf69" (UID: "daa28724-34bf-4c69-9bec-fcee00bc456f") : secret "metrics-daemon-secret" not found Jul 10 12:46:55.232601 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.232585 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 12:46:55.233933 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.233914 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ea5247d2-85e6-4959-9479-a6eea1212a2d-original-pull-secret\") pod \"global-pull-secret-syncer-8ln2b\" (UID: \"ea5247d2-85e6-4959-9479-a6eea1212a2d\") " pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:55.242492 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.242472 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmldp\" (UniqueName: \"kubernetes.io/projected/eb142d03-253f-4af6-97a8-80adc3e17656-kube-api-access-kmldp\") pod \"network-check-target-572wk\" (UID: \"eb142d03-253f-4af6-97a8-80adc3e17656\") " pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:55.320653 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.320625 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-stats-auth\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.320761 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.320658 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zggg\" (UniqueName: \"kubernetes.io/projected/77b92f38-8541-4c6c-b440-e05f745570ec-kube-api-access-4zggg\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.320761 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.320703 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-default-certificate\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.320761 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.320750 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:46:55.320918 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.320774 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.320918 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.320802 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.320918 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.320888 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 12:46:55.320918 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.320906 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dbb8b4668-6bfdw: secret "image-registry-tls" not found Jul 10 12:46:55.321082 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.320892 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 12:46:55.321082 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.320969 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:46:55.820942833 +0000 UTC m=+66.706703426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : configmap references non-existent config key: service-ca.crt Jul 10 12:46:55.321082 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.320989 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:46:55.820978976 +0000 UTC m=+66.706739575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : secret "router-metrics-certs-default" not found Jul 10 12:46:55.321082 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.321007 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls podName:4aec912a-9c70-489e-b0f2-1dce7f37e7ee nodeName:}" failed. No retries permitted until 2026-07-10 12:47:27.3209987 +0000 UTC m=+98.206759296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls") pod "image-registry-6dbb8b4668-6bfdw" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee") : secret "image-registry-tls" not found Jul 10 12:46:55.322733 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.322718 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-stats-auth\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.322959 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.322942 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-default-certificate\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.329232 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.329214 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zggg\" (UniqueName: \"kubernetes.io/projected/77b92f38-8541-4c6c-b440-e05f745570ec-kube-api-access-4zggg\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.421706 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.421641 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:46:55.421805 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.421745 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 12:46:55.421805 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.421792 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls podName:0694c98e-3ce3-45cb-8503-47b1acd27007 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:27.421779252 +0000 UTC m=+98.307539845 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls") pod "dns-default-p7l2z" (UID: "0694c98e-3ce3-45cb-8503-47b1acd27007") : secret "dns-default-metrics-tls" not found Jul 10 12:46:55.448753 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.448732 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8ln2b" Jul 10 12:46:55.456848 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.456828 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-99bwq\"" Jul 10 12:46:55.464909 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.464895 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:46:55.595928 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.595898 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8ln2b"] Jul 10 12:46:55.599484 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:46:55.599460 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5247d2_85e6_4959_9479_a6eea1212a2d.slice/crio-5c56f6eea51f6edf6d8e4ef3f93ef93b54a8f34ddf1d6933447141e04c777b80 WatchSource:0}: Error finding container 5c56f6eea51f6edf6d8e4ef3f93ef93b54a8f34ddf1d6933447141e04c777b80: Status 404 returned error can't find the container with id 5c56f6eea51f6edf6d8e4ef3f93ef93b54a8f34ddf1d6933447141e04c777b80 Jul 10 12:46:55.611363 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.611341 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-572wk"] Jul 10 12:46:55.613718 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:46:55.613697 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb142d03_253f_4af6_97a8_80adc3e17656.slice/crio-12638ae22c0dc59f4e563ac2e2391690f512f963f393e323868429c775e60440 WatchSource:0}: Error finding container 12638ae22c0dc59f4e563ac2e2391690f512f963f393e323868429c775e60440: Status 404 returned error can't find the container with id 12638ae22c0dc59f4e563ac2e2391690f512f963f393e323868429c775e60440 Jul 10 12:46:55.798931 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.798836 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-572wk" event={"ID":"eb142d03-253f-4af6-97a8-80adc3e17656","Type":"ContainerStarted","Data":"12638ae22c0dc59f4e563ac2e2391690f512f963f393e323868429c775e60440"} Jul 10 12:46:55.799799 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.799778 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8ln2b" event={"ID":"ea5247d2-85e6-4959-9479-a6eea1212a2d","Type":"ContainerStarted","Data":"5c56f6eea51f6edf6d8e4ef3f93ef93b54a8f34ddf1d6933447141e04c777b80"} Jul 10 12:46:55.826116 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.826095 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.826176 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:55.826122 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:55.826249 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.826235 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 12:46:55.826304 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.826240 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:46:56.826226738 +0000 UTC m=+67.711987332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : configmap references non-existent config key: service-ca.crt Jul 10 12:46:55.826353 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:55.826304 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:46:56.826276477 +0000 UTC m=+67.712037075 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : secret "router-metrics-certs-default" not found Jul 10 12:46:56.833401 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:56.833367 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:56.833833 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:56.833408 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:56.833833 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:56.833538 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 12:46:56.833833 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:56.833609 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:46:58.833590152 +0000 UTC m=+69.719350760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : configmap references non-existent config key: service-ca.crt Jul 10 12:46:56.833833 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:56.833632 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:46:58.833625913 +0000 UTC m=+69.719386506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : secret "router-metrics-certs-default" not found Jul 10 12:46:58.848529 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:58.848495 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:58.848951 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:58.848547 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:46:58.848951 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:58.848616 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 12:46:58.848951 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:58.848672 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:47:02.848658679 +0000 UTC m=+73.734419271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : configmap references non-existent config key: service-ca.crt Jul 10 12:46:58.848951 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:58.848689 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:47:02.848682026 +0000 UTC m=+73.734442618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : secret "router-metrics-certs-default" not found Jul 10 12:46:59.576444 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.576410 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mrqt7"] Jul 10 12:46:59.590691 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.590668 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.593855 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.593569 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 12:46:59.593855 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.593622 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 12:46:59.594043 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.593900 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mrqt7"] Jul 10 12:46:59.594043 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.593928 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cc7d5\"" Jul 10 12:46:59.594043 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.593949 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 12:46:59.594043 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.594001 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 12:46:59.655311 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.655256 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.655471 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.655368 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgxxb\" (UniqueName: \"kubernetes.io/projected/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-api-access-hgxxb\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.655471 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.655426 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-crio-socket\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.655590 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.655556 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.655641 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.655597 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-data-volume\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.756694 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.756655 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgxxb\" (UniqueName: \"kubernetes.io/projected/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-api-access-hgxxb\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.756865 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.756730 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-crio-socket\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.756865 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.756811 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.756865 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.756838 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-data-volume\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.756865 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.756865 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.757081 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.756874 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-crio-socket\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.757081 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:59.756962 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 12:46:59.757081 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:59.756965 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 12:46:59.757081 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:59.757042 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:00.257006485 +0000 UTC m=+71.142767078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : configmap "kube-rbac-proxy" not found Jul 10 12:46:59.757081 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:46:59.757062 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:00.257052414 +0000 UTC m=+71.142813011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : secret "insights-runtime-extractor-tls" not found Jul 10 12:46:59.757304 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.757250 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-data-volume\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:46:59.765744 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:46:59.765719 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgxxb\" (UniqueName: \"kubernetes.io/projected/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-api-access-hgxxb\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:00.260633 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:00.260591 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:00.261107 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:00.260650 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:00.261107 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:00.260737 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 12:47:00.261107 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:00.260794 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 12:47:00.261107 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:00.260817 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:01.260796719 +0000 UTC m=+72.146557315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : configmap "kube-rbac-proxy" not found Jul 10 12:47:00.261107 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:00.260843 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:01.260826906 +0000 UTC m=+72.146587500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : secret "insights-runtime-extractor-tls" not found Jul 10 12:47:00.813298 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:00.813265 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-572wk" event={"ID":"eb142d03-253f-4af6-97a8-80adc3e17656","Type":"ContainerStarted","Data":"0856f7f2ad81adce55f35309737ad52e59ec42b885a359cf7fced05753b3ff52"} Jul 10 12:47:00.813533 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:00.813372 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:47:00.814853 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:00.814833 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8ln2b" event={"ID":"ea5247d2-85e6-4959-9479-a6eea1212a2d","Type":"ContainerStarted","Data":"024b58bfd97ad7ce42d1995d04f086c52fc542e281659734d85421abed226554"} Jul 10 12:47:00.851961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:00.851912 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-572wk" podStartSLOduration=66.802851658 podStartE2EDuration="1m11.851897655s" totalImagesPullingTime="5.049045997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:00.834444507 +0000 UTC m=+71.720205119" watchObservedRunningTime="2026-07-10 12:47:00.851897655 +0000 UTC m=+71.737658288" Jul 10 12:47:00.852384 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:00.852358 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8ln2b" podStartSLOduration=66.78587976 podStartE2EDuration="1m11.85235316s" totalImagesPullingTime="5.0664734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:00.851420653 +0000 UTC m=+71.737181268" watchObservedRunningTime="2026-07-10 12:47:00.85235316 +0000 UTC m=+71.738113804" Jul 10 12:47:01.269852 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:01.269814 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:01.270271 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:01.269952 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:01.270271 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:01.269968 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 12:47:01.270271 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:01.270028 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:03.270012415 +0000 UTC m=+74.155773007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : secret "insights-runtime-extractor-tls" not found Jul 10 12:47:01.270271 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:01.270044 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 12:47:01.270271 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:01.270098 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:03.270083093 +0000 UTC m=+74.155843686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : configmap "kube-rbac-proxy" not found Jul 10 12:47:01.364522 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:01.364501 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6hl8t_4aeb355e-62c8-4c45-9b4a-795ab74cfe42/dns-node-resolver/0.log" Jul 10 12:47:02.364403 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:02.364378 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-84kdj_390a4dd4-04cc-4db6-93e5-33ee6222c93e/node-ca/0.log" Jul 10 12:47:02.881586 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:02.881542 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:02.881586 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:02.881589 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:02.881803 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:02.881689 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 12:47:02.881803 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:02.881705 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:47:10.881690578 +0000 UTC m=+81.767451170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : configmap references non-existent config key: service-ca.crt Jul 10 12:47:02.881803 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:02.881740 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:47:10.881726479 +0000 UTC m=+81.767487071 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : secret "router-metrics-certs-default" not found Jul 10 12:47:03.164839 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:03.164813 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prpw8_4f2f42e7-8190-41ef-a9a6-ec8b351dc05e/serve-healthcheck-canary/0.log" Jul 10 12:47:03.285238 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:03.285205 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:03.285386 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:03.285247 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:03.285386 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:03.285339 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 12:47:03.285465 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:03.285392 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 12:47:03.285465 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:03.285400 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:07.285385517 +0000 UTC m=+78.171146110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : configmap "kube-rbac-proxy" not found Jul 10 12:47:03.285465 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:03.285446 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:07.285434135 +0000 UTC m=+78.171194727 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : secret "insights-runtime-extractor-tls" not found Jul 10 12:47:07.319050 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:07.319010 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:07.319507 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:07.319059 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:07.319507 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:07.319132 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 12:47:07.319507 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:07.319176 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 12:47:07.319507 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:07.319203 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:15.319184906 +0000 UTC m=+86.204945511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : configmap "kube-rbac-proxy" not found Jul 10 12:47:07.319507 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:07.319240 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:15.319226195 +0000 UTC m=+86.204986792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : secret "insights-runtime-extractor-tls" not found Jul 10 12:47:10.947698 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:10.947559 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:10.947698 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:10.947611 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:10.948178 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:10.947723 2571 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 12:47:10.948178 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:10.947730 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:47:26.94771027 +0000 UTC m=+97.833470882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : configmap references non-existent config key: service-ca.crt Jul 10 12:47:10.948178 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:10.947785 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs podName:77b92f38-8541-4c6c-b440-e05f745570ec nodeName:}" failed. No retries permitted until 2026-07-10 12:47:26.947771228 +0000 UTC m=+97.833531820 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs") pod "router-default-794657f478-2bsbk" (UID: "77b92f38-8541-4c6c-b440-e05f745570ec") : secret "router-metrics-certs-default" not found Jul 10 12:47:15.378817 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:15.378772 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:15.378817 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:15.378824 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:15.379284 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:15.378916 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 12:47:15.379284 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:15.379018 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls podName:49dc2bdb-3536-46a3-9dff-3483bd8aa4e1 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:31.379000941 +0000 UTC m=+102.264761534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrqt7" (UID: "49dc2bdb-3536-46a3-9dff-3483bd8aa4e1") : secret "insights-runtime-extractor-tls" not found Jul 10 12:47:15.379384 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:15.379358 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:26.969485 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:26.969441 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:26.969485 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:26.969487 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:26.970179 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:26.970160 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77b92f38-8541-4c6c-b440-e05f745570ec-service-ca-bundle\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:26.971828 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:26.971808 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77b92f38-8541-4c6c-b440-e05f745570ec-metrics-certs\") pod \"router-default-794657f478-2bsbk\" (UID: \"77b92f38-8541-4c6c-b440-e05f745570ec\") " pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:27.260628 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.260520 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:27.372921 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.372892 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:47:27.375074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.375051 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"image-registry-6dbb8b4668-6bfdw\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:47:27.396753 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.396728 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-794657f478-2bsbk"] Jul 10 12:47:27.399988 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:27.399966 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b92f38_8541_4c6c_b440_e05f745570ec.slice/crio-897876bfab13fd5ed76402289a767f8bd3d08c427d89a776e35ede0d06e6089d WatchSource:0}: Error finding container 897876bfab13fd5ed76402289a767f8bd3d08c427d89a776e35ede0d06e6089d: Status 404 returned error can't find the container with id 897876bfab13fd5ed76402289a767f8bd3d08c427d89a776e35ede0d06e6089d Jul 10 12:47:27.423284 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.423267 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ch6zw\"" Jul 10 12:47:27.431347 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.431335 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:47:27.473661 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.473627 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:47:27.475913 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.475868 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0694c98e-3ce3-45cb-8503-47b1acd27007-metrics-tls\") pod \"dns-default-p7l2z\" (UID: \"0694c98e-3ce3-45cb-8503-47b1acd27007\") " pod="openshift-dns/dns-default-p7l2z" Jul 10 12:47:27.559282 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.559211 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dbb8b4668-6bfdw"] Jul 10 12:47:27.562115 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:27.562091 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aec912a_9c70_489e_b0f2_1dce7f37e7ee.slice/crio-ef1923559c5606bb6b6387decb041ab33f4058f14932910ddb43c87bf8a6a2dd WatchSource:0}: Error finding container ef1923559c5606bb6b6387decb041ab33f4058f14932910ddb43c87bf8a6a2dd: Status 404 returned error can't find the container with id ef1923559c5606bb6b6387decb041ab33f4058f14932910ddb43c87bf8a6a2dd Jul 10 12:47:27.737489 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.737465 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qfssw\"" Jul 10 12:47:27.745726 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.745710 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p7l2z" Jul 10 12:47:27.873946 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.873918 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p7l2z"] Jul 10 12:47:27.879200 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:27.879174 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0694c98e_3ce3_45cb_8503_47b1acd27007.slice/crio-bcda73ae08fd51943744fc0e21e3e47d0c3d1bef3afc2b3a136756afb3a820c5 WatchSource:0}: Error finding container bcda73ae08fd51943744fc0e21e3e47d0c3d1bef3afc2b3a136756afb3a820c5: Status 404 returned error can't find the container with id bcda73ae08fd51943744fc0e21e3e47d0c3d1bef3afc2b3a136756afb3a820c5 Jul 10 12:47:27.883022 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.882989 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-794657f478-2bsbk" event={"ID":"77b92f38-8541-4c6c-b440-e05f745570ec","Type":"ContainerStarted","Data":"893b9136df87058245442b3f804c3621f7a12a054199a8deb8d1423dd3af47f2"} Jul 10 12:47:27.883144 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.883044 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-794657f478-2bsbk" event={"ID":"77b92f38-8541-4c6c-b440-e05f745570ec","Type":"ContainerStarted","Data":"897876bfab13fd5ed76402289a767f8bd3d08c427d89a776e35ede0d06e6089d"} Jul 10 12:47:27.884717 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.884695 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" event={"ID":"4aec912a-9c70-489e-b0f2-1dce7f37e7ee","Type":"ContainerStarted","Data":"a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8"} Jul 10 12:47:27.884805 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.884732 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" event={"ID":"4aec912a-9c70-489e-b0f2-1dce7f37e7ee","Type":"ContainerStarted","Data":"ef1923559c5606bb6b6387decb041ab33f4058f14932910ddb43c87bf8a6a2dd"} Jul 10 12:47:27.884840 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.884822 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:47:27.900928 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.900870 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-794657f478-2bsbk" podStartSLOduration=32.900851086 podStartE2EDuration="32.900851086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:46:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:27.90005921 +0000 UTC m=+98.785819816" watchObservedRunningTime="2026-07-10 12:47:27.900851086 +0000 UTC m=+98.786611700" Jul 10 12:47:27.918391 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:27.918355 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" podStartSLOduration=98.918343869 podStartE2EDuration="1m38.918343869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:27.91771604 +0000 UTC m=+98.803476658" watchObservedRunningTime="2026-07-10 12:47:27.918343869 +0000 UTC m=+98.804104501" Jul 10 12:47:28.261284 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:28.261248 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:28.263640 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:28.263617 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:28.888701 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:28.888669 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p7l2z" event={"ID":"0694c98e-3ce3-45cb-8503-47b1acd27007","Type":"ContainerStarted","Data":"bcda73ae08fd51943744fc0e21e3e47d0c3d1bef3afc2b3a136756afb3a820c5"} Jul 10 12:47:28.888966 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:28.888945 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:28.890126 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:28.890101 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-794657f478-2bsbk" Jul 10 12:47:30.901198 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:30.901167 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p7l2z" event={"ID":"0694c98e-3ce3-45cb-8503-47b1acd27007","Type":"ContainerStarted","Data":"bd8c5ab8738bac33483fc2d223cfdba965956afd059aa911386e9085e511a497"} Jul 10 12:47:30.901656 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:30.901258 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p7l2z" event={"ID":"0694c98e-3ce3-45cb-8503-47b1acd27007","Type":"ContainerStarted","Data":"a4db93851888eda751f53e681dc06f8cc3f1c59d3188a15f69e782ab3ddfd9ce"} Jul 10 12:47:30.919352 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:30.919304 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-p7l2z" podStartSLOduration=65.896006203 podStartE2EDuration="1m7.919274322s" totalImagesPullingTime="2.023268119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:46:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:30.919054796 +0000 UTC m=+101.804815429" watchObservedRunningTime="2026-07-10 12:47:30.919274322 +0000 UTC m=+101.805035077" Jul 10 12:47:31.406649 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:31.406613 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:31.408803 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:31.408774 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/49dc2bdb-3536-46a3-9dff-3483bd8aa4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrqt7\" (UID: \"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1\") " pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:31.703082 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:31.702984 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mrqt7" Jul 10 12:47:31.819485 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:31.819462 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-572wk" Jul 10 12:47:31.853752 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:31.853723 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mrqt7"] Jul 10 12:47:31.856795 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:31.856772 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49dc2bdb_3536_46a3_9dff_3483bd8aa4e1.slice/crio-d8a54a2fc51bc0ab1f9b343ecb031a89a8041641d36f80ed29610904fecff52c WatchSource:0}: Error finding container d8a54a2fc51bc0ab1f9b343ecb031a89a8041641d36f80ed29610904fecff52c: Status 404 returned error can't find the container with id d8a54a2fc51bc0ab1f9b343ecb031a89a8041641d36f80ed29610904fecff52c Jul 10 12:47:31.905306 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:31.905265 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mrqt7" event={"ID":"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1","Type":"ContainerStarted","Data":"d8a54a2fc51bc0ab1f9b343ecb031a89a8041641d36f80ed29610904fecff52c"} Jul 10 12:47:31.905607 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:31.905460 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p7l2z" Jul 10 12:47:33.046923 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.046888 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98"] Jul 10 12:47:33.067975 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.067948 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-xsnvv"] Jul 10 12:47:33.068107 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.068058 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" Jul 10 12:47:33.071548 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.071522 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ng7pg\"" Jul 10 12:47:33.072397 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.072377 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 10 12:47:33.088097 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.088079 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-xzc8p"] Jul 10 12:47:33.088224 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.088208 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-xsnvv" Jul 10 12:47:33.096547 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.096367 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 12:47:33.096547 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.096413 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-xjv82\"" Jul 10 12:47:33.096733 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.096683 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 12:47:33.103166 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.103144 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98"] Jul 10 12:47:33.103280 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.103185 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-xsnvv"] Jul 10 12:47:33.103280 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.103206 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-xzc8p"] Jul 10 12:47:33.103424 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.103310 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" Jul 10 12:47:33.106389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.106251 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 10 12:47:33.106389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.106275 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 10 12:47:33.106552 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.106401 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-24qfr\"" Jul 10 12:47:33.220987 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.220952 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0e2fa4c-7ed3-431c-b438-cacc3838ff8b-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6hv98\" (UID: \"b0e2fa4c-7ed3-431c-b438-cacc3838ff8b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" Jul 10 12:47:33.221169 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.221003 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/00b769db-9621-4725-8449-04103e7c94ec-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xzc8p\" (UID: \"00b769db-9621-4725-8449-04103e7c94ec\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" Jul 10 12:47:33.221169 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.221041 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/00b769db-9621-4725-8449-04103e7c94ec-nginx-conf\") pod \"networking-console-plugin-6cf856758c-xzc8p\" (UID: \"00b769db-9621-4725-8449-04103e7c94ec\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" Jul 10 12:47:33.221169 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.221069 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zwpx\" (UniqueName: \"kubernetes.io/projected/af33c8f8-7312-4ff0-a28f-153f8f48e5d8-kube-api-access-9zwpx\") pod \"downloads-68c9cbb497-xsnvv\" (UID: \"af33c8f8-7312-4ff0-a28f-153f8f48e5d8\") " pod="openshift-console/downloads-68c9cbb497-xsnvv" Jul 10 12:47:33.321687 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.321608 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0e2fa4c-7ed3-431c-b438-cacc3838ff8b-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6hv98\" (UID: \"b0e2fa4c-7ed3-431c-b438-cacc3838ff8b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" Jul 10 12:47:33.321687 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.321643 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/00b769db-9621-4725-8449-04103e7c94ec-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xzc8p\" (UID: \"00b769db-9621-4725-8449-04103e7c94ec\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" Jul 10 12:47:33.321687 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.321673 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/00b769db-9621-4725-8449-04103e7c94ec-nginx-conf\") pod \"networking-console-plugin-6cf856758c-xzc8p\" (UID: \"00b769db-9621-4725-8449-04103e7c94ec\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" Jul 10 12:47:33.321955 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.321697 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zwpx\" (UniqueName: \"kubernetes.io/projected/af33c8f8-7312-4ff0-a28f-153f8f48e5d8-kube-api-access-9zwpx\") pod \"downloads-68c9cbb497-xsnvv\" (UID: \"af33c8f8-7312-4ff0-a28f-153f8f48e5d8\") " pod="openshift-console/downloads-68c9cbb497-xsnvv" Jul 10 12:47:33.324531 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.322756 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/00b769db-9621-4725-8449-04103e7c94ec-nginx-conf\") pod \"networking-console-plugin-6cf856758c-xzc8p\" (UID: \"00b769db-9621-4725-8449-04103e7c94ec\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" Jul 10 12:47:33.326136 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.325040 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/00b769db-9621-4725-8449-04103e7c94ec-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-xzc8p\" (UID: \"00b769db-9621-4725-8449-04103e7c94ec\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" Jul 10 12:47:33.326595 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.326569 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b0e2fa4c-7ed3-431c-b438-cacc3838ff8b-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6hv98\" (UID: \"b0e2fa4c-7ed3-431c-b438-cacc3838ff8b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" Jul 10 12:47:33.331030 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.331011 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zwpx\" (UniqueName: \"kubernetes.io/projected/af33c8f8-7312-4ff0-a28f-153f8f48e5d8-kube-api-access-9zwpx\") pod \"downloads-68c9cbb497-xsnvv\" (UID: \"af33c8f8-7312-4ff0-a28f-153f8f48e5d8\") " pod="openshift-console/downloads-68c9cbb497-xsnvv" Jul 10 12:47:33.378787 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.378755 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" Jul 10 12:47:33.400743 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.400723 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-xsnvv" Jul 10 12:47:33.414076 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.414049 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" Jul 10 12:47:33.568980 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.568947 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98"] Jul 10 12:47:33.572765 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:33.572708 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0e2fa4c_7ed3_431c_b438_cacc3838ff8b.slice/crio-4a96e90687ca7cc3ea57d5eda3f87757ea5dace94b2bf2b167ea10b7206dd8f3 WatchSource:0}: Error finding container 4a96e90687ca7cc3ea57d5eda3f87757ea5dace94b2bf2b167ea10b7206dd8f3: Status 404 returned error can't find the container with id 4a96e90687ca7cc3ea57d5eda3f87757ea5dace94b2bf2b167ea10b7206dd8f3 Jul 10 12:47:33.591512 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.591445 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-xsnvv"] Jul 10 12:47:33.594819 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:33.594794 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf33c8f8_7312_4ff0_a28f_153f8f48e5d8.slice/crio-639fa1efad96dd3a9c22a60585fef5284dbc4fa6c59232ee42bfd1d1cc3901f3 WatchSource:0}: Error finding container 639fa1efad96dd3a9c22a60585fef5284dbc4fa6c59232ee42bfd1d1cc3901f3: Status 404 returned error can't find the container with id 639fa1efad96dd3a9c22a60585fef5284dbc4fa6c59232ee42bfd1d1cc3901f3 Jul 10 12:47:33.616806 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.616782 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-xzc8p"] Jul 10 12:47:33.620575 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:33.620549 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b769db_9621_4725_8449_04103e7c94ec.slice/crio-97bde2aa89f13a6bba799ee5a3780f78b7e726683b22efbe0613775e21208c6a WatchSource:0}: Error finding container 97bde2aa89f13a6bba799ee5a3780f78b7e726683b22efbe0613775e21208c6a: Status 404 returned error can't find the container with id 97bde2aa89f13a6bba799ee5a3780f78b7e726683b22efbe0613775e21208c6a Jul 10 12:47:33.913881 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.913841 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" event={"ID":"00b769db-9621-4725-8449-04103e7c94ec","Type":"ContainerStarted","Data":"97bde2aa89f13a6bba799ee5a3780f78b7e726683b22efbe0613775e21208c6a"} Jul 10 12:47:33.915061 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.915028 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-xsnvv" event={"ID":"af33c8f8-7312-4ff0-a28f-153f8f48e5d8","Type":"ContainerStarted","Data":"639fa1efad96dd3a9c22a60585fef5284dbc4fa6c59232ee42bfd1d1cc3901f3"} Jul 10 12:47:33.916151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:33.916125 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" event={"ID":"b0e2fa4c-7ed3-431c-b438-cacc3838ff8b","Type":"ContainerStarted","Data":"4a96e90687ca7cc3ea57d5eda3f87757ea5dace94b2bf2b167ea10b7206dd8f3"} Jul 10 12:47:36.545944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.545907 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c44fd5447-mq6qk"] Jul 10 12:47:36.552172 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.552126 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.556106 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.555872 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zm5ns\"" Jul 10 12:47:36.556106 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.555891 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 12:47:36.556106 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.555974 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 12:47:36.556106 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.556075 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 12:47:36.556400 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.556127 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 12:47:36.556400 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.556181 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 12:47:36.559907 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.559883 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c44fd5447-mq6qk"] Jul 10 12:47:36.648747 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.648709 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-service-ca\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.648926 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.648762 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-serving-cert\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.648926 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.648814 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-oauth-config\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.648926 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.648842 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trj6t\" (UniqueName: \"kubernetes.io/projected/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-kube-api-access-trj6t\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.649090 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.648941 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-oauth-serving-cert\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.649090 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.648999 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-config\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.749485 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.749449 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-service-ca\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.749597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.749494 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-serving-cert\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.749597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.749527 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-oauth-config\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.749597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.749553 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trj6t\" (UniqueName: \"kubernetes.io/projected/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-kube-api-access-trj6t\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.749597 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.749582 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-oauth-serving-cert\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.749812 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.749614 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-config\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.751086 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.750441 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-config\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.751086 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.750556 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-service-ca\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.751086 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.751043 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-oauth-serving-cert\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.753539 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.753498 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-serving-cert\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.753997 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.753976 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-oauth-config\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.759049 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.759026 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trj6t\" (UniqueName: \"kubernetes.io/projected/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-kube-api-access-trj6t\") pod \"console-6c44fd5447-mq6qk\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.864876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.864842 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:36.930373 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.930274 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mrqt7" event={"ID":"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1","Type":"ContainerStarted","Data":"8a58020a34bfbba3655f0df69313c3e6d7416e760443031e91d7b42246103155"} Jul 10 12:47:36.932778 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.932711 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" event={"ID":"00b769db-9621-4725-8449-04103e7c94ec","Type":"ContainerStarted","Data":"3176876a26d96ecf6c536966d85cc9ba2cfeab6b34f096c0ee0e3230059dd0bf"} Jul 10 12:47:36.938540 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.935892 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" event={"ID":"b0e2fa4c-7ed3-431c-b438-cacc3838ff8b","Type":"ContainerStarted","Data":"0874fdeda5d405897b9a56362c746c6baced8224b445f83275a08712bb54c586"} Jul 10 12:47:36.938540 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.936513 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" Jul 10 12:47:36.942074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.942056 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" Jul 10 12:47:36.949944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.949858 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-xzc8p" podStartSLOduration=0.871575062 podStartE2EDuration="3.949841771s" totalImagesPullingTime="3.078266709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:36.948400391 +0000 UTC m=+107.834161016" watchObservedRunningTime="2026-07-10 12:47:36.949841771 +0000 UTC m=+107.835602385" Jul 10 12:47:36.964731 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:36.963738 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6hv98" podStartSLOduration=0.839584811 podStartE2EDuration="3.963718465s" totalImagesPullingTime="3.124133654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:36.96297664 +0000 UTC m=+107.848737256" watchObservedRunningTime="2026-07-10 12:47:36.963718465 +0000 UTC m=+107.849479081" Jul 10 12:47:37.014478 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.010606 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c44fd5447-mq6qk"] Jul 10 12:47:37.020748 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:37.020719 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9b3c1e_876e_45f3_b0bc_b874eb7ad3e1.slice/crio-b060b86ac1714f289fa2d739db0e0f6cf72bd916d6807c201d1ce3f9989dc3d7 WatchSource:0}: Error finding container b060b86ac1714f289fa2d739db0e0f6cf72bd916d6807c201d1ce3f9989dc3d7: Status 404 returned error can't find the container with id b060b86ac1714f289fa2d739db0e0f6cf72bd916d6807c201d1ce3f9989dc3d7 Jul 10 12:47:37.228305 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.228220 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rd6rk"] Jul 10 12:47:37.231267 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.231244 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.234087 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.234044 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 12:47:37.234087 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.234071 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7d656\"" Jul 10 12:47:37.234415 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.234050 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 12:47:37.234415 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.234072 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 12:47:37.234415 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.234044 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 12:47:37.234415 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.234205 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 12:47:37.241521 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.241503 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rd6rk"] Jul 10 12:47:37.354785 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.354736 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f225da2-9370-456b-84c5-a5f65bf83660-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.354964 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.354799 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f225da2-9370-456b-84c5-a5f65bf83660-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.354964 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.354942 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f225da2-9370-456b-84c5-a5f65bf83660-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.355081 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.354985 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6twwb\" (UniqueName: \"kubernetes.io/projected/0f225da2-9370-456b-84c5-a5f65bf83660-kube-api-access-6twwb\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.456386 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.456361 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f225da2-9370-456b-84c5-a5f65bf83660-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.456511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.456403 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6twwb\" (UniqueName: \"kubernetes.io/projected/0f225da2-9370-456b-84c5-a5f65bf83660-kube-api-access-6twwb\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.456511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.456455 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f225da2-9370-456b-84c5-a5f65bf83660-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.456511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.456484 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f225da2-9370-456b-84c5-a5f65bf83660-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.456671 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:37.456607 2571 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 10 12:47:37.456671 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:37.456668 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f225da2-9370-456b-84c5-a5f65bf83660-prometheus-operator-tls podName:0f225da2-9370-456b-84c5-a5f65bf83660 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:37.956650016 +0000 UTC m=+108.842410622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/0f225da2-9370-456b-84c5-a5f65bf83660-prometheus-operator-tls") pod "prometheus-operator-64785b6675-rd6rk" (UID: "0f225da2-9370-456b-84c5-a5f65bf83660") : secret "prometheus-operator-tls" not found Jul 10 12:47:37.457511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.457463 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f225da2-9370-456b-84c5-a5f65bf83660-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.459258 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.459234 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f225da2-9370-456b-84c5-a5f65bf83660-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.465859 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.465836 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6twwb\" (UniqueName: \"kubernetes.io/projected/0f225da2-9370-456b-84c5-a5f65bf83660-kube-api-access-6twwb\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.943680 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.943585 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mrqt7" event={"ID":"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1","Type":"ContainerStarted","Data":"d6dc22e7c4ff9d13b56fe879c1bd70e36241c43f0e69f2689873394fd7a4b1a3"} Jul 10 12:47:37.946001 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.945968 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c44fd5447-mq6qk" event={"ID":"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1","Type":"ContainerStarted","Data":"b060b86ac1714f289fa2d739db0e0f6cf72bd916d6807c201d1ce3f9989dc3d7"} Jul 10 12:47:37.959738 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.959709 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f225da2-9370-456b-84c5-a5f65bf83660-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:37.963162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:37.963133 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0f225da2-9370-456b-84c5-a5f65bf83660-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rd6rk\" (UID: \"0f225da2-9370-456b-84c5-a5f65bf83660\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:38.141965 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:38.141699 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" Jul 10 12:47:38.303952 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:38.303895 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rd6rk"] Jul 10 12:47:38.310945 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:38.310916 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f225da2_9370_456b_84c5_a5f65bf83660.slice/crio-658a8948ae1e491729b5f4f42401f89c4216e6a2b70e87b9c16f9fb1bab72c4a WatchSource:0}: Error finding container 658a8948ae1e491729b5f4f42401f89c4216e6a2b70e87b9c16f9fb1bab72c4a: Status 404 returned error can't find the container with id 658a8948ae1e491729b5f4f42401f89c4216e6a2b70e87b9c16f9fb1bab72c4a Jul 10 12:47:38.954020 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:38.953986 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" event={"ID":"0f225da2-9370-456b-84c5-a5f65bf83660","Type":"ContainerStarted","Data":"658a8948ae1e491729b5f4f42401f89c4216e6a2b70e87b9c16f9fb1bab72c4a"} Jul 10 12:47:41.911402 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:41.911354 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p7l2z" Jul 10 12:47:42.974105 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:42.974066 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mrqt7" event={"ID":"49dc2bdb-3536-46a3-9dff-3483bd8aa4e1","Type":"ContainerStarted","Data":"599d04d9ac2c87c76f96617d35b70f022f3e29f683f88521b8c62b2d161b8ca0"} Jul 10 12:47:42.976167 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:42.976048 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" event={"ID":"0f225da2-9370-456b-84c5-a5f65bf83660","Type":"ContainerStarted","Data":"42fdc1aef5e3a6108d5a99b55012ce8777db1a1bf60fb058e2687b01075b4ac7"} Jul 10 12:47:42.976167 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:42.976102 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" event={"ID":"0f225da2-9370-456b-84c5-a5f65bf83660","Type":"ContainerStarted","Data":"d799e47c67f19d032586b0599bb7c360d7ec19567b142ecb0defe3f2038af25c"} Jul 10 12:47:42.977614 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:42.977590 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c44fd5447-mq6qk" event={"ID":"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1","Type":"ContainerStarted","Data":"cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955"} Jul 10 12:47:42.992558 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:42.992436 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mrqt7" podStartSLOduration=33.913548151 podStartE2EDuration="43.992422844s" totalImagesPullingTime="10.078874693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 12:46:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:42.991164153 +0000 UTC m=+113.876924801" watchObservedRunningTime="2026-07-10 12:47:42.992422844 +0000 UTC m=+113.878183459" Jul 10 12:47:43.008506 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:43.008466 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c44fd5447-mq6qk" podStartSLOduration=1.9221072879999999 podStartE2EDuration="7.008453215s" totalImagesPullingTime="5.086345927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:43.007105442 +0000 UTC m=+113.892866057" watchObservedRunningTime="2026-07-10 12:47:43.008453215 +0000 UTC m=+113.894213830" Jul 10 12:47:43.024876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:43.024836 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-rd6rk" podStartSLOduration=2.233217769 podStartE2EDuration="6.024821147s" totalImagesPullingTime="3.791603378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:43.023645024 +0000 UTC m=+113.909405640" watchObservedRunningTime="2026-07-10 12:47:43.024821147 +0000 UTC m=+113.910581763" Jul 10 12:47:44.551221 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.551185 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv"] Jul 10 12:47:44.555079 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.555049 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.557800 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.557775 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 10 12:47:44.558312 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.558142 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 10 12:47:44.558411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.558315 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zchvx\"" Jul 10 12:47:44.567824 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.567805 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv"] Jul 10 12:47:44.580886 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.580864 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-hfmlp"] Jul 10 12:47:44.584037 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.584013 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.586516 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.586486 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 10 12:47:44.586757 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.586733 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 10 12:47:44.586887 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.586861 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 10 12:47:44.586960 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.586947 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-f8rmr\"" Jul 10 12:47:44.598875 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.598849 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rkv8j"] Jul 10 12:47:44.603635 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.603616 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-hfmlp"] Jul 10 12:47:44.603735 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.603719 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.606267 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.606248 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 12:47:44.606579 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.606553 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 12:47:44.606579 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.606563 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hw9ng\"" Jul 10 12:47:44.606579 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.606573 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 12:47:44.623629 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623592 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2270b672-089c-4691-a432-c7a20ef79918-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.623744 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623635 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.623744 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623664 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.623744 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623711 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct99m\" (UniqueName: \"kubernetes.io/projected/2270b672-089c-4691-a432-c7a20ef79918-kube-api-access-ct99m\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.623875 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623743 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2270b672-089c-4691-a432-c7a20ef79918-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.623875 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623776 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-accelerators-collector-config\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.623875 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623811 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.623875 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623836 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c5b5c42-b040-42a0-8d7d-074af02ae819-metrics-client-ca\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.624054 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623881 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-tls\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.624054 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623911 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vzg4\" (UniqueName: \"kubernetes.io/projected/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-api-access-9vzg4\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.624054 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.623960 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5c5b5c42-b040-42a0-8d7d-074af02ae819-root\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.624204 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.624051 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.624204 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.624078 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftwdx\" (UniqueName: \"kubernetes.io/projected/5c5b5c42-b040-42a0-8d7d-074af02ae819-kube-api-access-ftwdx\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.624204 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.624109 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-textfile\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.624204 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.624135 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-wtmp\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.624204 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.624171 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2270b672-089c-4691-a432-c7a20ef79918-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.624204 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.624194 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c5b5c42-b040-42a0-8d7d-074af02ae819-sys\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.624459 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.624257 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.624459 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.624280 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.725436 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725399 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-textfile\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.725664 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725445 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-wtmp\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.725664 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725480 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2270b672-089c-4691-a432-c7a20ef79918-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.725664 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725503 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c5b5c42-b040-42a0-8d7d-074af02ae819-sys\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.725664 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725545 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.725664 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725568 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.725664 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725591 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2270b672-089c-4691-a432-c7a20ef79918-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.725664 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725615 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.725664 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725643 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725672 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-wtmp\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725678 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct99m\" (UniqueName: \"kubernetes.io/projected/2270b672-089c-4691-a432-c7a20ef79918-kube-api-access-ct99m\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725762 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2270b672-089c-4691-a432-c7a20ef79918-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725781 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-textfile\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725806 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-accelerators-collector-config\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725852 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725879 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c5b5c42-b040-42a0-8d7d-074af02ae819-metrics-client-ca\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725914 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-tls\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725942 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vzg4\" (UniqueName: \"kubernetes.io/projected/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-api-access-9vzg4\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.725965 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5c5b5c42-b040-42a0-8d7d-074af02ae819-root\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.726021 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.726162 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.726046 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftwdx\" (UniqueName: \"kubernetes.io/projected/5c5b5c42-b040-42a0-8d7d-074af02ae819-kube-api-access-ftwdx\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.727007 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.726470 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2270b672-089c-4691-a432-c7a20ef79918-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.727007 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.726573 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5c5b5c42-b040-42a0-8d7d-074af02ae819-root\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.727007 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.726575 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.727007 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.726639 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c5b5c42-b040-42a0-8d7d-074af02ae819-sys\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.727007 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.726886 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.727350 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.727041 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.727466 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.727385 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c5b5c42-b040-42a0-8d7d-074af02ae819-metrics-client-ca\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.727553 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.727517 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-accelerators-collector-config\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.728966 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.728938 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.729185 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.729144 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.729410 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.729375 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.729510 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.729461 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2270b672-089c-4691-a432-c7a20ef79918-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.730752 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.730721 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2270b672-089c-4691-a432-c7a20ef79918-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.731107 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.731087 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c5b5c42-b040-42a0-8d7d-074af02ae819-node-exporter-tls\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.735928 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.735890 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftwdx\" (UniqueName: \"kubernetes.io/projected/5c5b5c42-b040-42a0-8d7d-074af02ae819-kube-api-access-ftwdx\") pod \"node-exporter-rkv8j\" (UID: \"5c5b5c42-b040-42a0-8d7d-074af02ae819\") " pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.738029 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.738008 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct99m\" (UniqueName: \"kubernetes.io/projected/2270b672-089c-4691-a432-c7a20ef79918-kube-api-access-ct99m\") pod \"openshift-state-metrics-84bd68b896-pftgv\" (UID: \"2270b672-089c-4691-a432-c7a20ef79918\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.738105 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.738080 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vzg4\" (UniqueName: \"kubernetes.io/projected/5abfdd0d-eebd-4f32-9d6d-04497a2f2bed-kube-api-access-9vzg4\") pod \"kube-state-metrics-785778ff74-hfmlp\" (UID: \"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.866490 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.866404 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" Jul 10 12:47:44.897792 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.897763 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" Jul 10 12:47:44.914138 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.914098 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rkv8j" Jul 10 12:47:44.987682 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:44.987643 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkv8j" event={"ID":"5c5b5c42-b040-42a0-8d7d-074af02ae819","Type":"ContainerStarted","Data":"3dd1fa2faa970a85a2fe2538e91031bf97e4dbde1f1666591c677f14a9e304f8"} Jul 10 12:47:45.019993 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.019918 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv"] Jul 10 12:47:45.022495 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:45.022466 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2270b672_089c_4691_a432_c7a20ef79918.slice/crio-bbbb2ab54f48e7554daa0e114dfab7e05f44699aae3e59e10fc2025a702d6053 WatchSource:0}: Error finding container bbbb2ab54f48e7554daa0e114dfab7e05f44699aae3e59e10fc2025a702d6053: Status 404 returned error can't find the container with id bbbb2ab54f48e7554daa0e114dfab7e05f44699aae3e59e10fc2025a702d6053 Jul 10 12:47:45.048500 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.048477 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-hfmlp"] Jul 10 12:47:45.052466 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:45.052437 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5abfdd0d_eebd_4f32_9d6d_04497a2f2bed.slice/crio-54b388a87dc63708535362eba5bea6541f775c9ec789adff01c4a5fdcb99f822 WatchSource:0}: Error finding container 54b388a87dc63708535362eba5bea6541f775c9ec789adff01c4a5fdcb99f822: Status 404 returned error can't find the container with id 54b388a87dc63708535362eba5bea6541f775c9ec789adff01c4a5fdcb99f822 Jul 10 12:47:45.633151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.633117 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 12:47:45.640174 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.640146 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.642915 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.642881 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 12:47:45.643062 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.642889 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 12:47:45.643062 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.643013 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 12:47:45.643448 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.643428 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 12:47:45.643556 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.643466 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9pdq8\"" Jul 10 12:47:45.643556 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.643493 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 12:47:45.643702 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.643689 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 12:47:45.643766 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.643723 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 12:47:45.644607 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.644512 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 12:47:45.644607 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.644516 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 12:47:45.655180 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.655146 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 12:47:45.732660 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732626 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-out\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.732835 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732664 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-volume\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.732835 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732687 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.732835 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732727 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.732835 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732774 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.732835 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732819 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.733074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732880 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.733074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732914 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.733074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732957 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbvb6\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-kube-api-access-jbvb6\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.733074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.732984 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.733074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.733025 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.733074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.733054 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-web-config\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.733259 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.733082 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834515 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834478 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834699 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834540 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-out\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834699 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834584 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-volume\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834699 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834605 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834699 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834644 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834699 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834672 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834704 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834729 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834783 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834839 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbvb6\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-kube-api-access-jbvb6\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834871 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834907 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.834961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.834938 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-web-config\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.835381 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:47:45.835318 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-trusted-ca-bundle podName:9063df0b-35bc-4934-ac06-3bd5b102ea87 nodeName:}" failed. No retries permitted until 2026-07-10 12:47:46.335282275 +0000 UTC m=+117.221042882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87") : configmap references non-existent config key: ca-bundle.crt Jul 10 12:47:45.836492 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.836468 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.836601 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.836468 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.838033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.838013 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-out\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.838284 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.838234 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.838610 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.838591 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-volume\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.839006 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.838991 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.839584 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.839560 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.840078 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.840031 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.840078 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.840050 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-web-config\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.840078 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.840031 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.840259 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.840233 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.844129 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.844110 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbvb6\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-kube-api-access-jbvb6\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:45.992611 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.992569 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" event={"ID":"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed","Type":"ContainerStarted","Data":"54b388a87dc63708535362eba5bea6541f775c9ec789adff01c4a5fdcb99f822"} Jul 10 12:47:45.994752 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.994706 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" event={"ID":"2270b672-089c-4691-a432-c7a20ef79918","Type":"ContainerStarted","Data":"0ec369ceac455e109411da569a8e56a35fb02d897015e59c10b7979960dd2348"} Jul 10 12:47:45.994904 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.994764 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" event={"ID":"2270b672-089c-4691-a432-c7a20ef79918","Type":"ContainerStarted","Data":"83ed720c379d0b073cad07952c0181d32eb7918bfa8c5ae66a5a5bfaa9681433"} Jul 10 12:47:45.994904 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:45.994806 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" event={"ID":"2270b672-089c-4691-a432-c7a20ef79918","Type":"ContainerStarted","Data":"bbbb2ab54f48e7554daa0e114dfab7e05f44699aae3e59e10fc2025a702d6053"} Jul 10 12:47:46.340760 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:46.340678 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:46.341980 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:46.341951 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:46.555019 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:46.554986 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:47:46.865450 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:46.865409 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:46.865913 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:46.865528 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:46.871417 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:46.871398 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:47.003869 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.003831 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:47:47.651005 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.650967 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4"] Jul 10 12:47:47.655013 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.654985 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.657572 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.657546 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 10 12:47:47.657701 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.657584 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 10 12:47:47.657701 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.657584 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q8hbl\"" Jul 10 12:47:47.657902 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.657876 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 10 12:47:47.657902 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.657892 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 10 12:47:47.658083 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.657892 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 10 12:47:47.658083 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.657979 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-24hups0ph5lut\"" Jul 10 12:47:47.666274 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.666254 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4"] Jul 10 12:47:47.752817 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.752785 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-tls\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.752984 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.752841 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj498\" (UniqueName: \"kubernetes.io/projected/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-kube-api-access-vj498\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.752984 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.752930 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.752984 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.752972 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-grpc-tls\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.753118 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.753033 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.753118 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.753066 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.753206 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.753181 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.753261 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.753205 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-metrics-client-ca\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.854128 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.854084 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.854343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.854155 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-metrics-client-ca\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.854343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.854191 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-tls\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.854343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.854228 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj498\" (UniqueName: \"kubernetes.io/projected/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-kube-api-access-vj498\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.854343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.854254 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.854343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.854276 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-grpc-tls\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.854343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.854333 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.854652 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.854392 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.855454 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.855402 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-metrics-client-ca\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.857409 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.857364 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.857503 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.857418 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-tls\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.857503 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.857432 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-grpc-tls\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.857503 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.857474 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.857886 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.857860 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.858057 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.858035 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.862874 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.862829 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj498\" (UniqueName: \"kubernetes.io/projected/009b7f1e-d869-4bc8-a21c-fdfe21ec7d78-kube-api-access-vj498\") pod \"thanos-querier-8fc8ff5d5-5ddz4\" (UID: \"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78\") " pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:47.968208 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:47.968119 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:47:48.895151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:48.895124 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:47:49.800120 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.800088 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd"] Jul 10 12:47:49.804908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.804886 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.807941 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.807851 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 12:47:49.807941 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.807853 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 12:47:49.807941 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.807888 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 12:47:49.807941 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.807853 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 12:47:49.808246 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.807853 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-wb72b\"" Jul 10 12:47:49.808246 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.807856 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 12:47:49.813395 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.813240 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 12:47:49.815363 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.815345 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd"] Jul 10 12:47:49.873349 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.873318 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.873518 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.873375 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-metrics-client-ca\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.873574 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.873524 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb9nm\" (UniqueName: \"kubernetes.io/projected/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-kube-api-access-tb9nm\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.873574 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.873563 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-federate-client-tls\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.873667 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.873601 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-telemeter-client-tls\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.873714 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.873676 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-serving-certs-ca-bundle\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.873714 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.873702 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-secret-telemeter-client\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.873785 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.873767 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.974895 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.974860 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-serving-certs-ca-bundle\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.974895 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.974892 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-secret-telemeter-client\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.975098 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.974911 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.975098 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.974937 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.975200 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.975113 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-metrics-client-ca\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.975251 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.975228 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb9nm\" (UniqueName: \"kubernetes.io/projected/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-kube-api-access-tb9nm\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.975326 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.975274 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-federate-client-tls\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.975382 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.975332 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-telemeter-client-tls\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.975868 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.975822 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-serving-certs-ca-bundle\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.976137 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.976090 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-metrics-client-ca\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.976491 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.976468 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.977985 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.977942 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.978227 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.978207 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-telemeter-client-tls\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.978364 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.978342 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-secret-telemeter-client\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.978438 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.978398 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-federate-client-tls\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:49.984218 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:49.984191 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb9nm\" (UniqueName: \"kubernetes.io/projected/4ee8cb5a-53aa-4894-a754-0cb6ecd988e8-kube-api-access-tb9nm\") pod \"telemeter-client-5dfdc7d66d-srdpd\" (UID: \"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8\") " pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:50.116315 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:50.116206 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" Jul 10 12:47:52.524906 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:52.524849 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4"] Jul 10 12:47:52.533465 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:52.533438 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod009b7f1e_d869_4bc8_a21c_fdfe21ec7d78.slice/crio-bbc6e2bf362c73dced939e16c1fd9526528148f5e15549ba70115ff7a476cb2a WatchSource:0}: Error finding container bbc6e2bf362c73dced939e16c1fd9526528148f5e15549ba70115ff7a476cb2a: Status 404 returned error can't find the container with id bbc6e2bf362c73dced939e16c1fd9526528148f5e15549ba70115ff7a476cb2a Jul 10 12:47:52.550510 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:52.550484 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 12:47:52.553701 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:52.553675 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9063df0b_35bc_4934_ac06_3bd5b102ea87.slice/crio-e261d2c74519abc413283136e055e10d2bccbd4ee79632e5c725fbc95c9cd820 WatchSource:0}: Error finding container e261d2c74519abc413283136e055e10d2bccbd4ee79632e5c725fbc95c9cd820: Status 404 returned error can't find the container with id e261d2c74519abc413283136e055e10d2bccbd4ee79632e5c725fbc95c9cd820 Jul 10 12:47:52.560888 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:52.560861 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd"] Jul 10 12:47:52.564885 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:52.564860 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ee8cb5a_53aa_4894_a754_0cb6ecd988e8.slice/crio-83b66b044117939621fec09e73f0b00a08c074d5849cf085270f29a88a29a140 WatchSource:0}: Error finding container 83b66b044117939621fec09e73f0b00a08c074d5849cf085270f29a88a29a140: Status 404 returned error can't find the container with id 83b66b044117939621fec09e73f0b00a08c074d5849cf085270f29a88a29a140 Jul 10 12:47:53.026158 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.026055 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" event={"ID":"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8","Type":"ContainerStarted","Data":"83b66b044117939621fec09e73f0b00a08c074d5849cf085270f29a88a29a140"} Jul 10 12:47:53.032338 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.031174 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-xsnvv" event={"ID":"af33c8f8-7312-4ff0-a28f-153f8f48e5d8","Type":"ContainerStarted","Data":"e8ad46465e3ce7d0cc3890792dbc9d6893ba4d5b44a6bb2455072c7058bf830b"} Jul 10 12:47:53.032338 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.032264 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-xsnvv" Jul 10 12:47:53.038479 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.038448 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerStarted","Data":"e261d2c74519abc413283136e055e10d2bccbd4ee79632e5c725fbc95c9cd820"} Jul 10 12:47:53.040584 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.040556 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" event={"ID":"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78","Type":"ContainerStarted","Data":"bbc6e2bf362c73dced939e16c1fd9526528148f5e15549ba70115ff7a476cb2a"} Jul 10 12:47:53.046511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.046487 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-xsnvv" Jul 10 12:47:53.055250 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.054453 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-xsnvv" podStartSLOduration=1.160082112 podStartE2EDuration="20.054434141s" totalImagesPullingTime="18.894352029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:53.053062571 +0000 UTC m=+123.938823189" watchObservedRunningTime="2026-07-10 12:47:53.054434141 +0000 UTC m=+123.940194759" Jul 10 12:47:53.581876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.581763 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dcf75fccf-q9lfv"] Jul 10 12:47:53.588724 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.588695 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.603321 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.602923 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dcf75fccf-q9lfv"] Jul 10 12:47:53.603929 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.603899 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 12:47:53.716572 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.716534 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-oauth-serving-cert\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.716775 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.716610 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-config\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.716775 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.716650 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-serving-cert\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.716775 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.716672 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4zzr\" (UniqueName: \"kubernetes.io/projected/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-kube-api-access-z4zzr\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.716775 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.716727 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-trusted-ca-bundle\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.716969 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.716777 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-service-ca\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.716969 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.716804 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-oauth-config\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.818316 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.818197 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-config\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.818316 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.818258 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-serving-cert\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.818316 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.818283 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4zzr\" (UniqueName: \"kubernetes.io/projected/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-kube-api-access-z4zzr\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.818629 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.818359 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-trusted-ca-bundle\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.822511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.819621 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-config\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.822511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.819709 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-service-ca\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.822511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.819742 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-oauth-config\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.822511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.819811 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-oauth-serving-cert\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.822511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.820535 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-oauth-serving-cert\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.822511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.820670 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-trusted-ca-bundle\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.822511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.820862 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-service-ca\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.829757 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.829732 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-serving-cert\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.831078 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.830552 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-oauth-config\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.836965 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.836814 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4zzr\" (UniqueName: \"kubernetes.io/projected/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-kube-api-access-z4zzr\") pod \"console-6dcf75fccf-q9lfv\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:53.907926 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:53.907841 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:47:56.510540 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:56.510510 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dcf75fccf-q9lfv"] Jul 10 12:47:57.286000 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:47:57.285963 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ecc40d_c699_4286_8db0_be4d7fd4f58e.slice/crio-f4ce883cbe1cf9ec54ba389df04cbf1f72d12ed41051c3baf8c8f055fe33c41a WatchSource:0}: Error finding container f4ce883cbe1cf9ec54ba389df04cbf1f72d12ed41051c3baf8c8f055fe33c41a: Status 404 returned error can't find the container with id f4ce883cbe1cf9ec54ba389df04cbf1f72d12ed41051c3baf8c8f055fe33c41a Jul 10 12:47:58.071525 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.071434 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" event={"ID":"2270b672-089c-4691-a432-c7a20ef79918","Type":"ContainerStarted","Data":"d0c24ca272c14a95ba22133c27bc6a5d95e0d5648f5b437832094203031d9727"} Jul 10 12:47:58.074866 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.074301 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dcf75fccf-q9lfv" event={"ID":"f2ecc40d-c699-4286-8db0-be4d7fd4f58e","Type":"ContainerStarted","Data":"3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418"} Jul 10 12:47:58.074866 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.074349 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dcf75fccf-q9lfv" event={"ID":"f2ecc40d-c699-4286-8db0-be4d7fd4f58e","Type":"ContainerStarted","Data":"f4ce883cbe1cf9ec54ba389df04cbf1f72d12ed41051c3baf8c8f055fe33c41a"} Jul 10 12:47:58.079806 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.079633 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" event={"ID":"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8","Type":"ContainerStarted","Data":"a40a7640ece27f9d26078b8a2666dbd3c358c7b18715ba46ac16f41e4c933703"} Jul 10 12:47:58.079806 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.079681 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" event={"ID":"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8","Type":"ContainerStarted","Data":"1ac6a39321d00352e63980a9bc9c739058ef11dbd601fd46311ccd46a5943518"} Jul 10 12:47:58.079806 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.079713 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" event={"ID":"4ee8cb5a-53aa-4894-a754-0cb6ecd988e8","Type":"ContainerStarted","Data":"0a12c87aac27a0f6ba2f11390a52bcad8577fae28279d7f1deb5f71e5f248abc"} Jul 10 12:47:58.082127 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.081985 2571 generic.go:356] "Generic (PLEG): container finished" podID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerID="5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a" exitCode=0 Jul 10 12:47:58.082127 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.082062 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerDied","Data":"5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a"} Jul 10 12:47:58.086502 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.085737 2571 generic.go:356] "Generic (PLEG): container finished" podID="5c5b5c42-b040-42a0-8d7d-074af02ae819" containerID="ba965e3a29893042fceccb68b6632971833641b1faba68e83042753c84d266aa" exitCode=0 Jul 10 12:47:58.086502 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.085818 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkv8j" event={"ID":"5c5b5c42-b040-42a0-8d7d-074af02ae819","Type":"ContainerDied","Data":"ba965e3a29893042fceccb68b6632971833641b1faba68e83042753c84d266aa"} Jul 10 12:47:58.089897 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.089820 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" event={"ID":"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed","Type":"ContainerStarted","Data":"2a6a941a302416e112924f58b5c83d15962612315a5692169478e4c53b33d01c"} Jul 10 12:47:58.090060 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.090012 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" event={"ID":"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed","Type":"ContainerStarted","Data":"a06ad085ff9d50fe9148a601d7eddd55c52c0290382b6a410b5af48689af4997"} Jul 10 12:47:58.090324 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.090225 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" event={"ID":"5abfdd0d-eebd-4f32-9d6d-04497a2f2bed","Type":"ContainerStarted","Data":"1b0a974bf17861efe6ceea2c9d09cc8b5939fe4ab9807abb9706be9057c2b63f"} Jul 10 12:47:58.094721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.094677 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-pftgv" podStartSLOduration=2.184640568 podStartE2EDuration="14.094659687s" totalImagesPullingTime="11.910019119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:58.093325524 +0000 UTC m=+128.979086140" watchObservedRunningTime="2026-07-10 12:47:58.094659687 +0000 UTC m=+128.980420304" Jul 10 12:47:58.094830 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.094793 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" event={"ID":"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78","Type":"ContainerStarted","Data":"19be3ec2330061c4b0b3700f5ba5a8c5774f0878663ed2c061072403a47f8127"} Jul 10 12:47:58.094899 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.094876 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" event={"ID":"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78","Type":"ContainerStarted","Data":"74585d004c200e33aba51d5401933dcb4da98a0399914198cc33559d0c1bb193"} Jul 10 12:47:58.094949 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.094922 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" event={"ID":"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78","Type":"ContainerStarted","Data":"d0eff3dba14d25c1912339001a26757d9640d154b89f09f157d6e48f71966adc"} Jul 10 12:47:58.158304 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.158246 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-hfmlp" podStartSLOduration=1.739612972 podStartE2EDuration="14.158231496s" totalImagesPullingTime="12.418618524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:58.157079372 +0000 UTC m=+129.042839988" watchObservedRunningTime="2026-07-10 12:47:58.158231496 +0000 UTC m=+129.043992111" Jul 10 12:47:58.181349 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.181243 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5dfdc7d66d-srdpd" podStartSLOduration=4.230617963 podStartE2EDuration="9.181227882s" totalImagesPullingTime="4.950609919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:58.179158795 +0000 UTC m=+129.064919424" watchObservedRunningTime="2026-07-10 12:47:58.181227882 +0000 UTC m=+129.066988497" Jul 10 12:47:58.196697 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.196650 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dcf75fccf-q9lfv" podStartSLOduration=5.196632538 podStartE2EDuration="5.196632538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:58.196197441 +0000 UTC m=+129.081958081" watchObservedRunningTime="2026-07-10 12:47:58.196632538 +0000 UTC m=+129.082393154" Jul 10 12:47:58.742446 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:58.741673 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dbb8b4668-6bfdw"] Jul 10 12:47:59.110302 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:59.110251 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkv8j" event={"ID":"5c5b5c42-b040-42a0-8d7d-074af02ae819","Type":"ContainerStarted","Data":"b8ecceb7e625e81ede84cf5af1dcbaa11ff725b7ea3a4fa54df3b152f75fafe5"} Jul 10 12:47:59.110853 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:59.110350 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rkv8j" event={"ID":"5c5b5c42-b040-42a0-8d7d-074af02ae819","Type":"ContainerStarted","Data":"4750050b31668553b942d58da42ce5ff11e81b17df9ed1565bcecd88348e29eb"} Jul 10 12:47:59.134451 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:59.132720 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rkv8j" podStartSLOduration=2.798052311 podStartE2EDuration="15.132675222s" totalImagesPullingTime="12.142310276s" totalInitContainerRuntime="192.312635ms" isStatefulPod=true podCreationTimestamp="2026-07-10 12:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:47:59.13173932 +0000 UTC m=+130.017499936" watchObservedRunningTime="2026-07-10 12:47:59.132675222 +0000 UTC m=+130.018435841" Jul 10 12:47:59.285576 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:59.285535 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:47:59.289482 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:59.289457 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/daa28724-34bf-4c69-9bec-fcee00bc456f-metrics-certs\") pod \"network-metrics-daemon-frf69\" (UID: \"daa28724-34bf-4c69-9bec-fcee00bc456f\") " pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:47:59.344841 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:59.344809 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pmvx9\"" Jul 10 12:47:59.352931 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:59.352902 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-frf69" Jul 10 12:47:59.537825 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:59.537743 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-frf69"] Jul 10 12:47:59.621150 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:47:59.621069 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dcf75fccf-q9lfv"] Jul 10 12:48:00.082987 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:48:00.082951 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa28724_34bf_4c69_9bec_fcee00bc456f.slice/crio-730aca77a3c178646622efafb682a59fbc07e32923b3120952716b8b3015650f WatchSource:0}: Error finding container 730aca77a3c178646622efafb682a59fbc07e32923b3120952716b8b3015650f: Status 404 returned error can't find the container with id 730aca77a3c178646622efafb682a59fbc07e32923b3120952716b8b3015650f Jul 10 12:48:00.115354 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:00.115323 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-frf69" event={"ID":"daa28724-34bf-4c69-9bec-fcee00bc456f","Type":"ContainerStarted","Data":"730aca77a3c178646622efafb682a59fbc07e32923b3120952716b8b3015650f"} Jul 10 12:48:01.122085 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:01.122006 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerStarted","Data":"d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436"} Jul 10 12:48:01.122469 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:01.122141 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerStarted","Data":"a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414"} Jul 10 12:48:01.122469 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:01.122182 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerStarted","Data":"40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08"} Jul 10 12:48:01.122469 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:01.122215 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerStarted","Data":"bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f"} Jul 10 12:48:01.128918 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:01.128886 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" event={"ID":"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78","Type":"ContainerStarted","Data":"ea8bfe00f0b3558b03d9a26922ed9979bebe8e654dac068fa46ad9f73fa72f30"} Jul 10 12:48:01.129031 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:01.128953 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" event={"ID":"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78","Type":"ContainerStarted","Data":"bed52090011867496f69ffd4f495cab72f44c3839a8ff7e64a83b7ed3d5dbe84"} Jul 10 12:48:01.129031 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:01.128993 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" event={"ID":"009b7f1e-d869-4bc8-a21c-fdfe21ec7d78","Type":"ContainerStarted","Data":"85ae0b1e3ce62b9bf63ebcad7780ea060aad0f9dca52e3a9601324e2bd8a5ae0"} Jul 10 12:48:01.129130 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:01.129036 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:48:01.152477 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:01.152417 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" podStartSLOduration=7.127754363 podStartE2EDuration="14.152397318s" totalImagesPullingTime="7.024642955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:48:01.151091366 +0000 UTC m=+132.036851998" watchObservedRunningTime="2026-07-10 12:48:01.152397318 +0000 UTC m=+132.038157935" Jul 10 12:48:02.135709 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:02.135674 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerStarted","Data":"73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a"} Jul 10 12:48:02.136137 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:02.135746 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerStarted","Data":"c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9"} Jul 10 12:48:02.137416 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:02.137389 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-frf69" event={"ID":"daa28724-34bf-4c69-9bec-fcee00bc456f","Type":"ContainerStarted","Data":"7f7ea946f3bfaa337164e250a8a7c623116b954583d958b1ff9257c298aa2691"} Jul 10 12:48:02.137526 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:02.137431 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-frf69" event={"ID":"daa28724-34bf-4c69-9bec-fcee00bc456f","Type":"ContainerStarted","Data":"8b241bf53fbc568be049582a6acbff7eac0e452d4f40d646b3f5b13d7b4cdcb9"} Jul 10 12:48:02.143362 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:02.143340 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8fc8ff5d5-5ddz4" Jul 10 12:48:02.168860 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:02.168812 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=10.173952408 podStartE2EDuration="17.168796984s" totalImagesPullingTime="6.955160243s" totalInitContainerRuntime="39.684333ms" isStatefulPod=false podCreationTimestamp="2026-07-10 12:47:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:48:02.166574855 +0000 UTC m=+133.052335680" watchObservedRunningTime="2026-07-10 12:48:02.168796984 +0000 UTC m=+133.054557598" Jul 10 12:48:02.213708 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:02.213648 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-frf69" podStartSLOduration=131.588597743 podStartE2EDuration="2m13.21362651s" totalImagesPullingTime="1.625028767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:48:02.212965273 +0000 UTC m=+133.098725889" watchObservedRunningTime="2026-07-10 12:48:02.21362651 +0000 UTC m=+133.099387125" Jul 10 12:48:03.908352 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:03.908312 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:48:10.497709 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:10.497679 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c44fd5447-mq6qk"] Jul 10 12:48:10.527399 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:10.527373 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794657f478-2bsbk_77b92f38-8541-4c6c-b440-e05f745570ec/router/0.log" Jul 10 12:48:10.541050 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:10.541026 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prpw8_4f2f42e7-8190-41ef-a9a6-ec8b351dc05e/serve-healthcheck-canary/0.log" Jul 10 12:48:23.778250 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:23.778186 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" podUID="4aec912a-9c70-489e-b0f2-1dce7f37e7ee" containerName="registry" containerID="cri-o://a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8" gracePeriod=30 Jul 10 12:48:24.073618 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.073597 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:48:24.112215 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.111341 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-bound-sa-token\") pod \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " Jul 10 12:48:24.112215 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.111414 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-image-registry-private-configuration\") pod \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " Jul 10 12:48:24.112215 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.111479 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88jnn\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-kube-api-access-88jnn\") pod \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " Jul 10 12:48:24.112215 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.111508 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-certificates\") pod \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " Jul 10 12:48:24.112215 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.111531 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") pod \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " Jul 10 12:48:24.112215 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.111586 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-ca-trust-extracted\") pod \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " Jul 10 12:48:24.112215 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.111620 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-trusted-ca\") pod \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " Jul 10 12:48:24.112215 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.111646 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-installation-pull-secrets\") pod \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\" (UID: \"4aec912a-9c70-489e-b0f2-1dce7f37e7ee\") " Jul 10 12:48:24.113444 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.113104 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4aec912a-9c70-489e-b0f2-1dce7f37e7ee" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:48:24.114849 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.114816 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4aec912a-9c70-489e-b0f2-1dce7f37e7ee" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:48:24.122053 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.121987 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4aec912a-9c70-489e-b0f2-1dce7f37e7ee" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:48:24.122165 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.122081 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4aec912a-9c70-489e-b0f2-1dce7f37e7ee" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:48:24.122278 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.122257 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4aec912a-9c70-489e-b0f2-1dce7f37e7ee" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:48:24.123503 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.123478 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4aec912a-9c70-489e-b0f2-1dce7f37e7ee" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:48:24.123715 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.123681 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-kube-api-access-88jnn" (OuterVolumeSpecName: "kube-api-access-88jnn") pod "4aec912a-9c70-489e-b0f2-1dce7f37e7ee" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee"). InnerVolumeSpecName "kube-api-access-88jnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:48:24.125615 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.125590 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4aec912a-9c70-489e-b0f2-1dce7f37e7ee" (UID: "4aec912a-9c70-489e-b0f2-1dce7f37e7ee"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:48:24.213332 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.213283 2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-image-registry-private-configuration\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:24.213569 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.213548 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88jnn\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-kube-api-access-88jnn\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:24.213679 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.213666 2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-certificates\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:24.213772 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.213760 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-registry-tls\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:24.213859 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.213847 2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-ca-trust-extracted\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:24.213939 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.213928 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-trusted-ca\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:24.214021 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.214010 2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-installation-pull-secrets\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:24.214178 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.214164 2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4aec912a-9c70-489e-b0f2-1dce7f37e7ee-bound-sa-token\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:24.229275 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.229248 2571 generic.go:356] "Generic (PLEG): container finished" podID="4aec912a-9c70-489e-b0f2-1dce7f37e7ee" containerID="a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8" exitCode=0 Jul 10 12:48:24.229434 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.229319 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" Jul 10 12:48:24.229434 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.229359 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" event={"ID":"4aec912a-9c70-489e-b0f2-1dce7f37e7ee","Type":"ContainerDied","Data":"a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8"} Jul 10 12:48:24.229434 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.229408 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dbb8b4668-6bfdw" event={"ID":"4aec912a-9c70-489e-b0f2-1dce7f37e7ee","Type":"ContainerDied","Data":"ef1923559c5606bb6b6387decb041ab33f4058f14932910ddb43c87bf8a6a2dd"} Jul 10 12:48:24.229434 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.229428 2571 scope.go:117] "RemoveContainer" containerID="a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8" Jul 10 12:48:24.239913 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.239837 2571 scope.go:117] "RemoveContainer" containerID="a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8" Jul 10 12:48:24.240281 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:48:24.240197 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8\": container with ID starting with a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8 not found: ID does not exist" containerID="a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8" Jul 10 12:48:24.240281 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.240230 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8"} err="failed to get container status \"a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8\": rpc error: code = NotFound desc = could not find container \"a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8\": container with ID starting with a930fc789a251f750469161b29e2de81b9ee96e77b68780a0891511fa95bebe8 not found: ID does not exist" Jul 10 12:48:24.250983 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.250961 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dbb8b4668-6bfdw"] Jul 10 12:48:24.257679 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:24.257654 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6dbb8b4668-6bfdw"] Jul 10 12:48:25.140738 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.140678 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6dcf75fccf-q9lfv" podUID="f2ecc40d-c699-4286-8db0-be4d7fd4f58e" containerName="console" containerID="cri-o://3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418" gracePeriod=15 Jul 10 12:48:25.433772 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.433743 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:48:25.526517 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.526482 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-oauth-serving-cert\") pod \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " Jul 10 12:48:25.526715 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.526536 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-service-ca\") pod \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " Jul 10 12:48:25.526715 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.526566 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4zzr\" (UniqueName: \"kubernetes.io/projected/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-kube-api-access-z4zzr\") pod \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " Jul 10 12:48:25.526715 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.526680 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-oauth-config\") pod \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " Jul 10 12:48:25.526897 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.526720 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-config\") pod \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " Jul 10 12:48:25.526897 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.526745 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-serving-cert\") pod \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " Jul 10 12:48:25.526897 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.526814 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-trusted-ca-bundle\") pod \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\" (UID: \"f2ecc40d-c699-4286-8db0-be4d7fd4f58e\") " Jul 10 12:48:25.527039 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.526969 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-service-ca" (OuterVolumeSpecName: "service-ca") pod "f2ecc40d-c699-4286-8db0-be4d7fd4f58e" (UID: "f2ecc40d-c699-4286-8db0-be4d7fd4f58e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:48:25.527039 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.526922 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f2ecc40d-c699-4286-8db0-be4d7fd4f58e" (UID: "f2ecc40d-c699-4286-8db0-be4d7fd4f58e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:48:25.527135 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.527092 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-oauth-serving-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:25.527135 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.527096 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-config" (OuterVolumeSpecName: "console-config") pod "f2ecc40d-c699-4286-8db0-be4d7fd4f58e" (UID: "f2ecc40d-c699-4286-8db0-be4d7fd4f58e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:48:25.527244 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.527112 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-service-ca\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:25.527281 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.527267 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f2ecc40d-c699-4286-8db0-be4d7fd4f58e" (UID: "f2ecc40d-c699-4286-8db0-be4d7fd4f58e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:48:25.528927 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.528909 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-kube-api-access-z4zzr" (OuterVolumeSpecName: "kube-api-access-z4zzr") pod "f2ecc40d-c699-4286-8db0-be4d7fd4f58e" (UID: "f2ecc40d-c699-4286-8db0-be4d7fd4f58e"). InnerVolumeSpecName "kube-api-access-z4zzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:48:25.529137 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.529121 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f2ecc40d-c699-4286-8db0-be4d7fd4f58e" (UID: "f2ecc40d-c699-4286-8db0-be4d7fd4f58e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:48:25.529198 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.529148 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f2ecc40d-c699-4286-8db0-be4d7fd4f58e" (UID: "f2ecc40d-c699-4286-8db0-be4d7fd4f58e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:48:25.535048 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.535022 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aec912a-9c70-489e-b0f2-1dce7f37e7ee" path="/var/lib/kubelet/pods/4aec912a-9c70-489e-b0f2-1dce7f37e7ee/volumes" Jul 10 12:48:25.628571 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.628527 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4zzr\" (UniqueName: \"kubernetes.io/projected/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-kube-api-access-z4zzr\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:25.628571 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.628561 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-oauth-config\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:25.628571 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.628571 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-config\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:25.628571 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.628580 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-console-serving-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:25.628823 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:25.628589 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2ecc40d-c699-4286-8db0-be4d7fd4f58e-trusted-ca-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:26.240460 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:26.240424 2571 generic.go:356] "Generic (PLEG): container finished" podID="f2ecc40d-c699-4286-8db0-be4d7fd4f58e" containerID="3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418" exitCode=0 Jul 10 12:48:26.240861 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:26.240467 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dcf75fccf-q9lfv" Jul 10 12:48:26.240861 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:26.240471 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dcf75fccf-q9lfv" event={"ID":"f2ecc40d-c699-4286-8db0-be4d7fd4f58e","Type":"ContainerDied","Data":"3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418"} Jul 10 12:48:26.240861 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:26.240533 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dcf75fccf-q9lfv" event={"ID":"f2ecc40d-c699-4286-8db0-be4d7fd4f58e","Type":"ContainerDied","Data":"f4ce883cbe1cf9ec54ba389df04cbf1f72d12ed41051c3baf8c8f055fe33c41a"} Jul 10 12:48:26.240861 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:26.240562 2571 scope.go:117] "RemoveContainer" containerID="3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418" Jul 10 12:48:26.248763 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:26.248746 2571 scope.go:117] "RemoveContainer" containerID="3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418" Jul 10 12:48:26.249023 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:48:26.249004 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418\": container with ID starting with 3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418 not found: ID does not exist" containerID="3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418" Jul 10 12:48:26.249089 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:26.249032 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418"} err="failed to get container status \"3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418\": rpc error: code = NotFound desc = could not find container \"3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418\": container with ID starting with 3e2ce593b2c34b576d8235854498254864de72c4d20032731fda8a6314954418 not found: ID does not exist" Jul 10 12:48:26.258618 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:26.258588 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6dcf75fccf-q9lfv"] Jul 10 12:48:26.262667 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:26.262648 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6dcf75fccf-q9lfv"] Jul 10 12:48:27.536809 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:27.536781 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2ecc40d-c699-4286-8db0-be4d7fd4f58e" path="/var/lib/kubelet/pods/f2ecc40d-c699-4286-8db0-be4d7fd4f58e/volumes" Jul 10 12:48:35.518469 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.518408 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c44fd5447-mq6qk" podUID="0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" containerName="console" containerID="cri-o://cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955" gracePeriod=15 Jul 10 12:48:35.812150 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.812124 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:48:35.907681 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.907654 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-serving-cert\") pod \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " Jul 10 12:48:35.907681 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.907683 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-oauth-config\") pod \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " Jul 10 12:48:35.907842 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.907708 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-config\") pod \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " Jul 10 12:48:35.907888 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.907841 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-oauth-serving-cert\") pod \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " Jul 10 12:48:35.907943 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.907899 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trj6t\" (UniqueName: \"kubernetes.io/projected/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-kube-api-access-trj6t\") pod \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " Jul 10 12:48:35.907997 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.907946 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-service-ca\") pod \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\" (UID: \"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1\") " Jul 10 12:48:35.908178 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.908154 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-config" (OuterVolumeSpecName: "console-config") pod "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" (UID: "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:48:35.908286 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.908265 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" (UID: "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:48:35.908368 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.908328 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" (UID: "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:48:35.909839 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.909816 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" (UID: "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:48:35.909925 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.909856 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" (UID: "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:48:35.909966 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:35.909928 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-kube-api-access-trj6t" (OuterVolumeSpecName: "kube-api-access-trj6t") pod "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" (UID: "0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1"). InnerVolumeSpecName "kube-api-access-trj6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:48:36.009245 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.009219 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trj6t\" (UniqueName: \"kubernetes.io/projected/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-kube-api-access-trj6t\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:36.009245 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.009244 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-service-ca\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:36.009409 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.009253 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-serving-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:36.009409 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.009262 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-oauth-config\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:36.009409 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.009271 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-console-config\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:36.009409 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.009279 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1-oauth-serving-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:48:36.282033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.282000 2571 generic.go:356] "Generic (PLEG): container finished" podID="0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" containerID="cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955" exitCode=0 Jul 10 12:48:36.282180 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.282037 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c44fd5447-mq6qk" event={"ID":"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1","Type":"ContainerDied","Data":"cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955"} Jul 10 12:48:36.282180 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.282047 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c44fd5447-mq6qk" Jul 10 12:48:36.282180 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.282075 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c44fd5447-mq6qk" event={"ID":"0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1","Type":"ContainerDied","Data":"b060b86ac1714f289fa2d739db0e0f6cf72bd916d6807c201d1ce3f9989dc3d7"} Jul 10 12:48:36.282180 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.282095 2571 scope.go:117] "RemoveContainer" containerID="cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955" Jul 10 12:48:36.290168 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.290153 2571 scope.go:117] "RemoveContainer" containerID="cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955" Jul 10 12:48:36.290423 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:48:36.290403 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955\": container with ID starting with cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955 not found: ID does not exist" containerID="cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955" Jul 10 12:48:36.290489 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.290428 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955"} err="failed to get container status \"cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955\": rpc error: code = NotFound desc = could not find container \"cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955\": container with ID starting with cc5abc7c4480104fb2b122f276bf6b7d52422a2c90f49d4d5551ba2375b4c955 not found: ID does not exist" Jul 10 12:48:36.303075 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.303047 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c44fd5447-mq6qk"] Jul 10 12:48:36.306383 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:36.306365 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c44fd5447-mq6qk"] Jul 10 12:48:37.537138 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:48:37.537104 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" path="/var/lib/kubelet/pods/0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1/volumes" Jul 10 12:49:04.951819 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:04.951785 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 12:49:04.952370 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:04.952107 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="alertmanager" containerID="cri-o://bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f" gracePeriod=120 Jul 10 12:49:04.952370 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:04.952156 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy-metric" containerID="cri-o://c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9" gracePeriod=120 Jul 10 12:49:04.952370 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:04.952237 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="prom-label-proxy" containerID="cri-o://73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a" gracePeriod=120 Jul 10 12:49:04.952370 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:04.952281 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="config-reloader" containerID="cri-o://40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08" gracePeriod=120 Jul 10 12:49:04.952370 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:04.952237 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy-web" containerID="cri-o://a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414" gracePeriod=120 Jul 10 12:49:04.952633 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:04.952266 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy" containerID="cri-o://d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436" gracePeriod=120 Jul 10 12:49:05.400823 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:05.400795 2571 generic.go:356] "Generic (PLEG): container finished" podID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerID="73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a" exitCode=0 Jul 10 12:49:05.400823 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:05.400817 2571 generic.go:356] "Generic (PLEG): container finished" podID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerID="d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436" exitCode=0 Jul 10 12:49:05.400823 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:05.400823 2571 generic.go:356] "Generic (PLEG): container finished" podID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerID="40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08" exitCode=0 Jul 10 12:49:05.400823 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:05.400829 2571 generic.go:356] "Generic (PLEG): container finished" podID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerID="bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f" exitCode=0 Jul 10 12:49:05.401132 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:05.400872 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerDied","Data":"73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a"} Jul 10 12:49:05.401132 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:05.400950 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerDied","Data":"d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436"} Jul 10 12:49:05.401132 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:05.400989 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerDied","Data":"40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08"} Jul 10 12:49:05.401132 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:05.401028 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerDied","Data":"bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f"} Jul 10 12:49:06.192670 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.192648 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.341876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.341779 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-tls-assets\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.341876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.341827 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.341876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.341845 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-metrics-client-ca\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.341876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.341866 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-volume\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342161 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342002 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-web-config\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342161 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342051 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-trusted-ca-bundle\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342161 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342087 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-cluster-tls-config\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342161 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342116 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-main-tls\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342161 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342159 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-main-db\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342230 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbvb6\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-kube-api-access-jbvb6\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342237 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:49:06.342455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342271 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-web\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342329 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342364 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-out\") pod \"9063df0b-35bc-4934-ac06-3bd5b102ea87\" (UID: \"9063df0b-35bc-4934-ac06-3bd5b102ea87\") " Jul 10 12:49:06.342720 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342614 2571 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-metrics-client-ca\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.342773 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.342737 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:49:06.345174 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.344888 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:49:06.345323 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.345194 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:49:06.345323 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.345254 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:49:06.345323 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.345263 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-volume" (OuterVolumeSpecName: "config-volume") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:49:06.345323 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.345283 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-kube-api-access-jbvb6" (OuterVolumeSpecName: "kube-api-access-jbvb6") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "kube-api-access-jbvb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:49:06.345632 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.345502 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-out" (OuterVolumeSpecName: "config-out") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:49:06.345632 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.345510 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:49:06.346799 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.346777 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:49:06.347043 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.347022 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:49:06.348925 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.348901 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:49:06.356267 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.356243 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-web-config" (OuterVolumeSpecName: "web-config") pod "9063df0b-35bc-4934-ac06-3bd5b102ea87" (UID: "9063df0b-35bc-4934-ac06-3bd5b102ea87"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:49:06.407591 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.407562 2571 generic.go:356] "Generic (PLEG): container finished" podID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerID="c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9" exitCode=0 Jul 10 12:49:06.407591 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.407586 2571 generic.go:356] "Generic (PLEG): container finished" podID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerID="a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414" exitCode=0 Jul 10 12:49:06.407712 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.407642 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.407712 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.407654 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerDied","Data":"c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9"} Jul 10 12:49:06.407779 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.407718 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerDied","Data":"a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414"} Jul 10 12:49:06.407779 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.407747 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9063df0b-35bc-4934-ac06-3bd5b102ea87","Type":"ContainerDied","Data":"e261d2c74519abc413283136e055e10d2bccbd4ee79632e5c725fbc95c9cd820"} Jul 10 12:49:06.407779 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.407776 2571 scope.go:117] "RemoveContainer" containerID="73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a" Jul 10 12:49:06.420696 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.420677 2571 scope.go:117] "RemoveContainer" containerID="c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9" Jul 10 12:49:06.427556 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.427539 2571 scope.go:117] "RemoveContainer" containerID="d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436" Jul 10 12:49:06.434521 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.434499 2571 scope.go:117] "RemoveContainer" containerID="a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414" Jul 10 12:49:06.434736 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.434718 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 12:49:06.438826 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.438806 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 12:49:06.441766 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.441746 2571 scope.go:117] "RemoveContainer" containerID="40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08" Jul 10 12:49:06.443896 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443879 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbvb6\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-kube-api-access-jbvb6\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.443975 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443899 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.443975 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443910 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.443975 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443920 2571 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-out\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.443975 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443928 2571 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9063df0b-35bc-4934-ac06-3bd5b102ea87-tls-assets\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.443975 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443936 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.443975 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443944 2571 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-config-volume\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.443975 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443971 2571 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-web-config\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.444186 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443978 2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.444186 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443987 2571 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-cluster-tls-config\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.444186 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.443996 2571 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9063df0b-35bc-4934-ac06-3bd5b102ea87-secret-alertmanager-main-tls\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.444186 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.444004 2571 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9063df0b-35bc-4934-ac06-3bd5b102ea87-alertmanager-main-db\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:49:06.448192 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.448176 2571 scope.go:117] "RemoveContainer" containerID="bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f" Jul 10 12:49:06.455033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.455018 2571 scope.go:117] "RemoveContainer" containerID="5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a" Jul 10 12:49:06.461802 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.461788 2571 scope.go:117] "RemoveContainer" containerID="73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a" Jul 10 12:49:06.462043 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:49:06.462025 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a\": container with ID starting with 73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a not found: ID does not exist" containerID="73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a" Jul 10 12:49:06.462085 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.462053 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a"} err="failed to get container status \"73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a\": rpc error: code = NotFound desc = could not find container \"73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a\": container with ID starting with 73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a not found: ID does not exist" Jul 10 12:49:06.462085 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.462071 2571 scope.go:117] "RemoveContainer" containerID="c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9" Jul 10 12:49:06.462355 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:49:06.462332 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9\": container with ID starting with c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9 not found: ID does not exist" containerID="c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9" Jul 10 12:49:06.462402 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.462365 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9"} err="failed to get container status \"c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9\": rpc error: code = NotFound desc = could not find container \"c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9\": container with ID starting with c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9 not found: ID does not exist" Jul 10 12:49:06.462402 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.462387 2571 scope.go:117] "RemoveContainer" containerID="d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436" Jul 10 12:49:06.462633 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:49:06.462616 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436\": container with ID starting with d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436 not found: ID does not exist" containerID="d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436" Jul 10 12:49:06.462673 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.462651 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436"} err="failed to get container status \"d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436\": rpc error: code = NotFound desc = could not find container \"d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436\": container with ID starting with d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436 not found: ID does not exist" Jul 10 12:49:06.462673 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.462665 2571 scope.go:117] "RemoveContainer" containerID="a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414" Jul 10 12:49:06.462894 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:49:06.462879 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414\": container with ID starting with a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414 not found: ID does not exist" containerID="a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414" Jul 10 12:49:06.462934 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.462898 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414"} err="failed to get container status \"a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414\": rpc error: code = NotFound desc = could not find container \"a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414\": container with ID starting with a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414 not found: ID does not exist" Jul 10 12:49:06.462934 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.462912 2571 scope.go:117] "RemoveContainer" containerID="40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08" Jul 10 12:49:06.463223 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:49:06.463162 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08\": container with ID starting with 40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08 not found: ID does not exist" containerID="40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08" Jul 10 12:49:06.463223 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.463197 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08"} err="failed to get container status \"40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08\": rpc error: code = NotFound desc = could not find container \"40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08\": container with ID starting with 40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08 not found: ID does not exist" Jul 10 12:49:06.463223 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.463219 2571 scope.go:117] "RemoveContainer" containerID="bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f" Jul 10 12:49:06.464005 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:49:06.463980 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f\": container with ID starting with bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f not found: ID does not exist" containerID="bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f" Jul 10 12:49:06.464099 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.464012 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f"} err="failed to get container status \"bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f\": rpc error: code = NotFound desc = could not find container \"bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f\": container with ID starting with bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f not found: ID does not exist" Jul 10 12:49:06.464099 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.464035 2571 scope.go:117] "RemoveContainer" containerID="5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a" Jul 10 12:49:06.464326 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:49:06.464284 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a\": container with ID starting with 5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a not found: ID does not exist" containerID="5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a" Jul 10 12:49:06.464404 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.464335 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a"} err="failed to get container status \"5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a\": rpc error: code = NotFound desc = could not find container \"5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a\": container with ID starting with 5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a not found: ID does not exist" Jul 10 12:49:06.464404 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.464353 2571 scope.go:117] "RemoveContainer" containerID="73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a" Jul 10 12:49:06.464585 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.464567 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a"} err="failed to get container status \"73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a\": rpc error: code = NotFound desc = could not find container \"73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a\": container with ID starting with 73fd2d64f001f58aaaa3694f4e43608036aeb969fcd8b108abc5232f44852f3a not found: ID does not exist" Jul 10 12:49:06.464627 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.464586 2571 scope.go:117] "RemoveContainer" containerID="c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9" Jul 10 12:49:06.464825 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.464804 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9"} err="failed to get container status \"c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9\": rpc error: code = NotFound desc = could not find container \"c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9\": container with ID starting with c1ab8f2432d2eb482e6078cf5c990d2fc7d0e393cce3833c03eda70f464138f9 not found: ID does not exist" Jul 10 12:49:06.464871 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.464828 2571 scope.go:117] "RemoveContainer" containerID="d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436" Jul 10 12:49:06.465062 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465044 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436"} err="failed to get container status \"d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436\": rpc error: code = NotFound desc = could not find container \"d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436\": container with ID starting with d3edc928041ffab777a39abb58d592e5d294dc46f510fffa386308ca805fc436 not found: ID does not exist" Jul 10 12:49:06.465111 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465064 2571 scope.go:117] "RemoveContainer" containerID="a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414" Jul 10 12:49:06.465329 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465309 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414"} err="failed to get container status \"a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414\": rpc error: code = NotFound desc = could not find container \"a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414\": container with ID starting with a49778785388ec7d1e3b6de8ba824565e2616c82442f2409ca5544544e024414 not found: ID does not exist" Jul 10 12:49:06.465376 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465330 2571 scope.go:117] "RemoveContainer" containerID="40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08" Jul 10 12:49:06.465554 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465531 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08"} err="failed to get container status \"40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08\": rpc error: code = NotFound desc = could not find container \"40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08\": container with ID starting with 40575a065a99863a7c5430abd5cec932f21d948b8035133c860cf39a4d7b6c08 not found: ID does not exist" Jul 10 12:49:06.465554 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465552 2571 scope.go:117] "RemoveContainer" containerID="bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f" Jul 10 12:49:06.465770 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465748 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f"} err="failed to get container status \"bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f\": rpc error: code = NotFound desc = could not find container \"bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f\": container with ID starting with bb52702470ada2d3b90d57a3c2c9927963a462d7e2637fc9aa5d8c245d13564f not found: ID does not exist" Jul 10 12:49:06.465817 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465772 2571 scope.go:117] "RemoveContainer" containerID="5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a" Jul 10 12:49:06.465882 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465866 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 12:49:06.465984 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.465965 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a"} err="failed to get container status \"5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a\": rpc error: code = NotFound desc = could not find container \"5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a\": container with ID starting with 5bf78498bb2469c4504272b59b041a19336e74b3f59b6f11a76fa079b4f25e9a not found: ID does not exist" Jul 10 12:49:06.466201 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466189 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy-metric" Jul 10 12:49:06.466236 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466203 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy-metric" Jul 10 12:49:06.466236 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466213 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy" Jul 10 12:49:06.466236 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466218 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy" Jul 10 12:49:06.466236 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466227 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy-web" Jul 10 12:49:06.466236 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466233 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy-web" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466240 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="init-config-reloader" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466246 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="init-config-reloader" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466256 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2ecc40d-c699-4286-8db0-be4d7fd4f58e" containerName="console" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466262 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2ecc40d-c699-4286-8db0-be4d7fd4f58e" containerName="console" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466268 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="prom-label-proxy" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466273 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="prom-label-proxy" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466280 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" containerName="console" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466285 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" containerName="console" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466309 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aec912a-9c70-489e-b0f2-1dce7f37e7ee" containerName="registry" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466314 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aec912a-9c70-489e-b0f2-1dce7f37e7ee" containerName="registry" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466322 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="alertmanager" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466327 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="alertmanager" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466333 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="config-reloader" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466337 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="config-reloader" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466384 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="prom-label-proxy" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466393 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="config-reloader" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466398 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2ecc40d-c699-4286-8db0-be4d7fd4f58e" containerName="console" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466403 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aec912a-9c70-489e-b0f2-1dce7f37e7ee" containerName="registry" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466409 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="alertmanager" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466415 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e9b3c1e-876e-45f3-b0bc-b874eb7ad3e1" containerName="console" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466419 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy-web" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466423 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy" Jul 10 12:49:06.466470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.466427 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" containerName="kube-rbac-proxy-metric" Jul 10 12:49:06.469541 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.469525 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.474532 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.474514 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 12:49:06.474644 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.474572 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 12:49:06.474711 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.474658 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 12:49:06.474837 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.474820 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 12:49:06.474927 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.474913 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 12:49:06.475114 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.475101 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 12:49:06.475201 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.475105 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 12:49:06.475682 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.475668 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9pdq8\"" Jul 10 12:49:06.476303 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.476271 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 12:49:06.482380 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.482365 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 12:49:06.485020 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.485003 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 12:49:06.645692 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645664 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb9dce00-6e9c-4ffe-90ce-91f300552654-config-out\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.645824 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645698 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.645824 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645716 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb9dce00-6e9c-4ffe-90ce-91f300552654-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.645824 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645781 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb9dce00-6e9c-4ffe-90ce-91f300552654-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.645935 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645834 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.645935 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645861 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb9dce00-6e9c-4ffe-90ce-91f300552654-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.645935 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645887 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-web-config\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.645935 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645924 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb9dce00-6e9c-4ffe-90ce-91f300552654-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.646052 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645948 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.646052 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.645974 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8lwq\" (UniqueName: \"kubernetes.io/projected/bb9dce00-6e9c-4ffe-90ce-91f300552654-kube-api-access-q8lwq\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.646052 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.646013 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.646052 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.646028 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.646052 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.646049 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747206 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747176 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb9dce00-6e9c-4ffe-90ce-91f300552654-config-out\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747206 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747204 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747223 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb9dce00-6e9c-4ffe-90ce-91f300552654-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747249 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb9dce00-6e9c-4ffe-90ce-91f300552654-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747277 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747318 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb9dce00-6e9c-4ffe-90ce-91f300552654-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747343 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-web-config\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747633 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747486 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb9dce00-6e9c-4ffe-90ce-91f300552654-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747633 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747535 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747633 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747567 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8lwq\" (UniqueName: \"kubernetes.io/projected/bb9dce00-6e9c-4ffe-90ce-91f300552654-kube-api-access-q8lwq\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747633 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747605 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747633 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747631 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.747877 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.747662 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.748209 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.748101 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb9dce00-6e9c-4ffe-90ce-91f300552654-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.748440 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.748400 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb9dce00-6e9c-4ffe-90ce-91f300552654-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.748682 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.748656 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb9dce00-6e9c-4ffe-90ce-91f300552654-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.749928 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.749904 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb9dce00-6e9c-4ffe-90ce-91f300552654-config-out\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.750028 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.749979 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-web-config\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.750308 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.750261 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.750472 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.750451 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb9dce00-6e9c-4ffe-90ce-91f300552654-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.750600 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.750564 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.750908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.750886 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.750953 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.750942 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.751260 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.751242 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.751851 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.751834 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb9dce00-6e9c-4ffe-90ce-91f300552654-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.756546 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.756527 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8lwq\" (UniqueName: \"kubernetes.io/projected/bb9dce00-6e9c-4ffe-90ce-91f300552654-kube-api-access-q8lwq\") pod \"alertmanager-main-0\" (UID: \"bb9dce00-6e9c-4ffe-90ce-91f300552654\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.779552 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.779531 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 12:49:06.907973 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:06.907804 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 12:49:06.910388 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:49:06.910358 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb9dce00_6e9c_4ffe_90ce_91f300552654.slice/crio-83a1f40f1e54e97f752e52e20c07944df9350cd8afef0be8fe2aa39af51f5e65 WatchSource:0}: Error finding container 83a1f40f1e54e97f752e52e20c07944df9350cd8afef0be8fe2aa39af51f5e65: Status 404 returned error can't find the container with id 83a1f40f1e54e97f752e52e20c07944df9350cd8afef0be8fe2aa39af51f5e65 Jul 10 12:49:07.412851 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:07.412813 2571 generic.go:356] "Generic (PLEG): container finished" podID="bb9dce00-6e9c-4ffe-90ce-91f300552654" containerID="e402c3484cd850b4b57d1317f654596f4c6a5114ba80a3b95173c10f1564dd03" exitCode=0 Jul 10 12:49:07.413252 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:07.412896 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb9dce00-6e9c-4ffe-90ce-91f300552654","Type":"ContainerDied","Data":"e402c3484cd850b4b57d1317f654596f4c6a5114ba80a3b95173c10f1564dd03"} Jul 10 12:49:07.413252 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:07.412962 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb9dce00-6e9c-4ffe-90ce-91f300552654","Type":"ContainerStarted","Data":"83a1f40f1e54e97f752e52e20c07944df9350cd8afef0be8fe2aa39af51f5e65"} Jul 10 12:49:07.533641 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:07.533615 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9063df0b-35bc-4934-ac06-3bd5b102ea87" path="/var/lib/kubelet/pods/9063df0b-35bc-4934-ac06-3bd5b102ea87/volumes" Jul 10 12:49:08.420977 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:08.420935 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb9dce00-6e9c-4ffe-90ce-91f300552654","Type":"ContainerStarted","Data":"feab976d807f9f40b9dc0401af84ec747a8baec3e416023e625cb68ffe933892"} Jul 10 12:49:08.421343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:08.421018 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb9dce00-6e9c-4ffe-90ce-91f300552654","Type":"ContainerStarted","Data":"e7acf0409d67e3fe4be554a8122fc42ee4229216a135afe561d63147153c3cf4"} Jul 10 12:49:08.421343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:08.421062 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb9dce00-6e9c-4ffe-90ce-91f300552654","Type":"ContainerStarted","Data":"fbd065ab43b4d97f7e586ace5d13f3ef0dad2257a4f808cbf752738c37fb97bd"} Jul 10 12:49:08.421343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:08.421105 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb9dce00-6e9c-4ffe-90ce-91f300552654","Type":"ContainerStarted","Data":"f242ee62a486bb0f11a8b886214ee3e02998ff2f4beb2a5d02bf91241e6f9f9d"} Jul 10 12:49:08.421343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:08.421138 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb9dce00-6e9c-4ffe-90ce-91f300552654","Type":"ContainerStarted","Data":"d455f1ab0407f843ef9102e99bb9ef79ddf575e7468e5728d29301d43ec1b956"} Jul 10 12:49:08.421343 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:08.421176 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb9dce00-6e9c-4ffe-90ce-91f300552654","Type":"ContainerStarted","Data":"3be217e2a3210aaf1640a4f7f77f0b082c3f839e52130604aada3fe3d43aefb8"} Jul 10 12:49:08.449816 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:08.449757 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.433384255 podStartE2EDuration="2.449742015s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.35776ms" isStatefulPod=false podCreationTimestamp="2026-07-10 12:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:49:08.446769411 +0000 UTC m=+199.332530027" watchObservedRunningTime="2026-07-10 12:49:08.449742015 +0000 UTC m=+199.335502630" Jul 10 12:49:13.484671 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.484638 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b8c6f76b-dr2d5"] Jul 10 12:49:13.486778 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.486761 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.490177 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.490153 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 12:49:13.490177 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.490173 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 12:49:13.490463 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.490446 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 12:49:13.490524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.490504 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zm5ns\"" Jul 10 12:49:13.490524 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.490518 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 12:49:13.491218 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.491197 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 12:49:13.495555 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.495535 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 12:49:13.501019 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.501000 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b8c6f76b-dr2d5"] Jul 10 12:49:13.604534 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.604503 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-config\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.604694 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.604553 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-serving-cert\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.604694 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.604583 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-trusted-ca-bundle\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.604694 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.604607 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-service-ca\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.604694 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.604637 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-oauth-serving-cert\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.604694 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.604670 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfc8t\" (UniqueName: \"kubernetes.io/projected/524ef394-21a4-4bef-bdec-c3728d7a94b3-kube-api-access-bfc8t\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.604694 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.604694 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-oauth-config\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.705784 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.705744 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfc8t\" (UniqueName: \"kubernetes.io/projected/524ef394-21a4-4bef-bdec-c3728d7a94b3-kube-api-access-bfc8t\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.705897 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.705795 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-oauth-config\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.705897 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.705828 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-config\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.705989 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.705958 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-serving-cert\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.706029 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.706004 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-trusted-ca-bundle\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.706029 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.706021 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-service-ca\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.706152 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.706133 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-oauth-serving-cert\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.706658 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.706624 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-config\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.706793 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.706676 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-service-ca\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.706793 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.706768 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-oauth-serving-cert\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.706958 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.706938 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-trusted-ca-bundle\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.708358 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.708330 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-serving-cert\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.708358 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.708343 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-oauth-config\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.715659 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.715638 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfc8t\" (UniqueName: \"kubernetes.io/projected/524ef394-21a4-4bef-bdec-c3728d7a94b3-kube-api-access-bfc8t\") pod \"console-77b8c6f76b-dr2d5\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.796983 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.796907 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:13.921190 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:13.921163 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b8c6f76b-dr2d5"] Jul 10 12:49:13.923719 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:49:13.923694 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524ef394_21a4_4bef_bdec_c3728d7a94b3.slice/crio-5bcd80d4cf6ed63eac56034c8944dd1568f5cd70307e8d95e5520f0e6584ae65 WatchSource:0}: Error finding container 5bcd80d4cf6ed63eac56034c8944dd1568f5cd70307e8d95e5520f0e6584ae65: Status 404 returned error can't find the container with id 5bcd80d4cf6ed63eac56034c8944dd1568f5cd70307e8d95e5520f0e6584ae65 Jul 10 12:49:14.446378 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:14.446338 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b8c6f76b-dr2d5" event={"ID":"524ef394-21a4-4bef-bdec-c3728d7a94b3","Type":"ContainerStarted","Data":"542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969"} Jul 10 12:49:14.446526 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:14.446450 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b8c6f76b-dr2d5" event={"ID":"524ef394-21a4-4bef-bdec-c3728d7a94b3","Type":"ContainerStarted","Data":"5bcd80d4cf6ed63eac56034c8944dd1568f5cd70307e8d95e5520f0e6584ae65"} Jul 10 12:49:14.464636 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:14.464593 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77b8c6f76b-dr2d5" podStartSLOduration=1.464578624 podStartE2EDuration="1.464578624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:49:14.463636268 +0000 UTC m=+205.349396884" watchObservedRunningTime="2026-07-10 12:49:14.464578624 +0000 UTC m=+205.350339238" Jul 10 12:49:23.797802 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:23.797759 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:23.798275 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:23.797821 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:23.802537 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:23.802514 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:49:24.489800 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:49:24.489777 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:50:08.242460 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.242431 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx"] Jul 10 12:50:08.245425 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.245408 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.248493 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.248474 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 12:50:08.248772 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.248756 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 12:50:08.249412 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.249393 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8p5fv\"" Jul 10 12:50:08.255423 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.255404 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx"] Jul 10 12:50:08.312305 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.312265 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.312411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.312314 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzltk\" (UniqueName: \"kubernetes.io/projected/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-kube-api-access-tzltk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.312411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.312367 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.412661 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.412636 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.412745 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.412667 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzltk\" (UniqueName: \"kubernetes.io/projected/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-kube-api-access-tzltk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.412745 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.412737 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.413083 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.413063 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.413122 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.413075 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.421189 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.421160 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzltk\" (UniqueName: \"kubernetes.io/projected/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-kube-api-access-tzltk\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.554260 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.554185 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:08.678325 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:08.678285 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx"] Jul 10 12:50:08.680073 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:50:08.680047 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa6da07a_c0e6_4ee4_93c6_0c041e47779c.slice/crio-5f1f19b690a354c3d35b7f69719962f9ba1b4d1094b967d51246d04c372649e1 WatchSource:0}: Error finding container 5f1f19b690a354c3d35b7f69719962f9ba1b4d1094b967d51246d04c372649e1: Status 404 returned error can't find the container with id 5f1f19b690a354c3d35b7f69719962f9ba1b4d1094b967d51246d04c372649e1 Jul 10 12:50:09.651354 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:09.651321 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" event={"ID":"fa6da07a-c0e6-4ee4-93c6-0c041e47779c","Type":"ContainerStarted","Data":"5f1f19b690a354c3d35b7f69719962f9ba1b4d1094b967d51246d04c372649e1"} Jul 10 12:50:15.683584 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:15.683505 2571 generic.go:356] "Generic (PLEG): container finished" podID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerID="2ca10b02cec5cba26913c02e17491951fbc093396fd5d847fdaa95dcd61eda95" exitCode=0 Jul 10 12:50:15.683584 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:15.683576 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" event={"ID":"fa6da07a-c0e6-4ee4-93c6-0c041e47779c","Type":"ContainerDied","Data":"2ca10b02cec5cba26913c02e17491951fbc093396fd5d847fdaa95dcd61eda95"} Jul 10 12:50:18.698030 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:18.698000 2571 generic.go:356] "Generic (PLEG): container finished" podID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerID="90395f50d00fa19da7a104d91764a199c662c82648475d5ee208be7b7299dc6d" exitCode=0 Jul 10 12:50:18.698430 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:18.698095 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" event={"ID":"fa6da07a-c0e6-4ee4-93c6-0c041e47779c","Type":"ContainerDied","Data":"90395f50d00fa19da7a104d91764a199c662c82648475d5ee208be7b7299dc6d"} Jul 10 12:50:26.737645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:26.737563 2571 generic.go:356] "Generic (PLEG): container finished" podID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerID="dde0b0b6959b23f1f8ecfc88b2a3f5a0f8da887a946b743b2b150be3b0b59dc0" exitCode=0 Jul 10 12:50:26.737997 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:26.737639 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" event={"ID":"fa6da07a-c0e6-4ee4-93c6-0c041e47779c","Type":"ContainerDied","Data":"dde0b0b6959b23f1f8ecfc88b2a3f5a0f8da887a946b743b2b150be3b0b59dc0"} Jul 10 12:50:27.864083 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:27.864063 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:27.974468 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:27.974446 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-bundle\") pod \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " Jul 10 12:50:27.974550 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:27.974485 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-util\") pod \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " Jul 10 12:50:27.974550 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:27.974516 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzltk\" (UniqueName: \"kubernetes.io/projected/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-kube-api-access-tzltk\") pod \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\" (UID: \"fa6da07a-c0e6-4ee4-93c6-0c041e47779c\") " Jul 10 12:50:27.975098 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:27.975076 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-bundle" (OuterVolumeSpecName: "bundle") pod "fa6da07a-c0e6-4ee4-93c6-0c041e47779c" (UID: "fa6da07a-c0e6-4ee4-93c6-0c041e47779c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:50:27.976590 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:27.976564 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-kube-api-access-tzltk" (OuterVolumeSpecName: "kube-api-access-tzltk") pod "fa6da07a-c0e6-4ee4-93c6-0c041e47779c" (UID: "fa6da07a-c0e6-4ee4-93c6-0c041e47779c"). InnerVolumeSpecName "kube-api-access-tzltk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:50:27.979585 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:27.979566 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-util" (OuterVolumeSpecName: "util") pod "fa6da07a-c0e6-4ee4-93c6-0c041e47779c" (UID: "fa6da07a-c0e6-4ee4-93c6-0c041e47779c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:50:28.076139 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:28.076084 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:50:28.076139 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:28.076104 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-util\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:50:28.076139 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:28.076113 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzltk\" (UniqueName: \"kubernetes.io/projected/fa6da07a-c0e6-4ee4-93c6-0c041e47779c-kube-api-access-tzltk\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:50:28.746211 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:28.746178 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" event={"ID":"fa6da07a-c0e6-4ee4-93c6-0c041e47779c","Type":"ContainerDied","Data":"5f1f19b690a354c3d35b7f69719962f9ba1b4d1094b967d51246d04c372649e1"} Jul 10 12:50:28.746211 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:28.746199 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540shckx" Jul 10 12:50:28.746431 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:28.746220 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f1f19b690a354c3d35b7f69719962f9ba1b4d1094b967d51246d04c372649e1" Jul 10 12:50:36.266629 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.266599 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6"] Jul 10 12:50:36.267066 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.266907 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerName="util" Jul 10 12:50:36.267066 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.266917 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerName="util" Jul 10 12:50:36.267066 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.266927 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerName="pull" Jul 10 12:50:36.267066 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.266932 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerName="pull" Jul 10 12:50:36.267066 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.266946 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerName="extract" Jul 10 12:50:36.267066 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.266952 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerName="extract" Jul 10 12:50:36.267066 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.267000 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa6da07a-c0e6-4ee4-93c6-0c041e47779c" containerName="extract" Jul 10 12:50:36.270156 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.270138 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" Jul 10 12:50:36.273523 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.273501 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 10 12:50:36.273655 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.273541 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 10 12:50:36.274994 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.274981 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-24ppt\"" Jul 10 12:50:36.290802 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.290778 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6"] Jul 10 12:50:36.337857 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.337820 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb3555bf-bae6-470e-a13b-7b0c327d9777-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-njtr6\" (UID: \"bb3555bf-bae6-470e-a13b-7b0c327d9777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" Jul 10 12:50:36.337976 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.337864 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmsm9\" (UniqueName: \"kubernetes.io/projected/bb3555bf-bae6-470e-a13b-7b0c327d9777-kube-api-access-rmsm9\") pod \"cert-manager-operator-controller-manager-55c444bf86-njtr6\" (UID: \"bb3555bf-bae6-470e-a13b-7b0c327d9777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" Jul 10 12:50:36.438860 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.438834 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb3555bf-bae6-470e-a13b-7b0c327d9777-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-njtr6\" (UID: \"bb3555bf-bae6-470e-a13b-7b0c327d9777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" Jul 10 12:50:36.438972 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.438864 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmsm9\" (UniqueName: \"kubernetes.io/projected/bb3555bf-bae6-470e-a13b-7b0c327d9777-kube-api-access-rmsm9\") pod \"cert-manager-operator-controller-manager-55c444bf86-njtr6\" (UID: \"bb3555bf-bae6-470e-a13b-7b0c327d9777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" Jul 10 12:50:36.439210 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.439188 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb3555bf-bae6-470e-a13b-7b0c327d9777-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-njtr6\" (UID: \"bb3555bf-bae6-470e-a13b-7b0c327d9777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" Jul 10 12:50:36.447404 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.447384 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmsm9\" (UniqueName: \"kubernetes.io/projected/bb3555bf-bae6-470e-a13b-7b0c327d9777-kube-api-access-rmsm9\") pod \"cert-manager-operator-controller-manager-55c444bf86-njtr6\" (UID: \"bb3555bf-bae6-470e-a13b-7b0c327d9777\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" Jul 10 12:50:36.579036 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.578967 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" Jul 10 12:50:36.709200 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.709175 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6"] Jul 10 12:50:36.711583 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:50:36.711557 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3555bf_bae6_470e_a13b_7b0c327d9777.slice/crio-9f51c79f2d9e6c2c5c70f702cc9207ec96e85a46782e52ed270716530c852a4f WatchSource:0}: Error finding container 9f51c79f2d9e6c2c5c70f702cc9207ec96e85a46782e52ed270716530c852a4f: Status 404 returned error can't find the container with id 9f51c79f2d9e6c2c5c70f702cc9207ec96e85a46782e52ed270716530c852a4f Jul 10 12:50:36.779090 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:36.779058 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" event={"ID":"bb3555bf-bae6-470e-a13b-7b0c327d9777","Type":"ContainerStarted","Data":"9f51c79f2d9e6c2c5c70f702cc9207ec96e85a46782e52ed270716530c852a4f"} Jul 10 12:50:39.792772 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:39.792736 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" event={"ID":"bb3555bf-bae6-470e-a13b-7b0c327d9777","Type":"ContainerStarted","Data":"6b73372e81e5cbe345df5864d67776797720c32ba41cb458b5ef05515be763c5"} Jul 10 12:50:39.815104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:39.815057 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-njtr6" podStartSLOduration=1.743959002 podStartE2EDuration="3.81503917s" totalImagesPullingTime="2.071080168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:50:39.813589929 +0000 UTC m=+290.699350535" watchObservedRunningTime="2026-07-10 12:50:39.81503917 +0000 UTC m=+290.700799790" Jul 10 12:50:40.653407 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.653375 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck"] Jul 10 12:50:40.657351 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.657334 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.660412 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.660390 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 12:50:40.660412 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.660398 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 12:50:40.660549 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.660525 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8p5fv\"" Jul 10 12:50:40.665559 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.665538 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck"] Jul 10 12:50:40.675349 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.675325 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.675472 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.675364 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ww5r\" (UniqueName: \"kubernetes.io/projected/a699b834-5ad8-472b-b71b-c2217c4e00b1-kube-api-access-2ww5r\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.675472 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.675405 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.776132 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.776110 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ww5r\" (UniqueName: \"kubernetes.io/projected/a699b834-5ad8-472b-b71b-c2217c4e00b1-kube-api-access-2ww5r\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.776246 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.776147 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.776246 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.776210 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.776541 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.776525 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.776584 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.776552 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.784926 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.784905 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ww5r\" (UniqueName: \"kubernetes.io/projected/a699b834-5ad8-472b-b71b-c2217c4e00b1-kube-api-access-2ww5r\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:40.967906 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:40.967819 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:41.095325 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:41.095301 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck"] Jul 10 12:50:41.097596 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:50:41.097570 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda699b834_5ad8_472b_b71b_c2217c4e00b1.slice/crio-84e7942e47479bdfd7d69d6e86e651d9ae48bc9656955b4448725b625dc76367 WatchSource:0}: Error finding container 84e7942e47479bdfd7d69d6e86e651d9ae48bc9656955b4448725b625dc76367: Status 404 returned error can't find the container with id 84e7942e47479bdfd7d69d6e86e651d9ae48bc9656955b4448725b625dc76367 Jul 10 12:50:41.802963 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:41.802935 2571 generic.go:356] "Generic (PLEG): container finished" podID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerID="d90c7b481a5abea4c9e272158529c9da0625827afa2822f5370fc57e8d5eeb24" exitCode=0 Jul 10 12:50:41.803092 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:41.802988 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" event={"ID":"a699b834-5ad8-472b-b71b-c2217c4e00b1","Type":"ContainerDied","Data":"d90c7b481a5abea4c9e272158529c9da0625827afa2822f5370fc57e8d5eeb24"} Jul 10 12:50:41.803092 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:41.803031 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" event={"ID":"a699b834-5ad8-472b-b71b-c2217c4e00b1","Type":"ContainerStarted","Data":"84e7942e47479bdfd7d69d6e86e651d9ae48bc9656955b4448725b625dc76367"} Jul 10 12:50:42.012918 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.012886 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v"] Jul 10 12:50:42.016042 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.016026 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" Jul 10 12:50:42.018560 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.018536 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 12:50:42.018657 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.018543 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mwxng\"" Jul 10 12:50:42.018657 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.018582 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 12:50:42.026045 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.026023 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v"] Jul 10 12:50:42.087389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.087332 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac58627b-09af-4b04-92ea-628bab81312a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vqx7v\" (UID: \"ac58627b-09af-4b04-92ea-628bab81312a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" Jul 10 12:50:42.087389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.087366 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cp8x\" (UniqueName: \"kubernetes.io/projected/ac58627b-09af-4b04-92ea-628bab81312a-kube-api-access-4cp8x\") pod \"cert-manager-cainjector-559cdd8c87-vqx7v\" (UID: \"ac58627b-09af-4b04-92ea-628bab81312a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" Jul 10 12:50:42.187737 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.187711 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac58627b-09af-4b04-92ea-628bab81312a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vqx7v\" (UID: \"ac58627b-09af-4b04-92ea-628bab81312a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" Jul 10 12:50:42.187859 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.187744 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cp8x\" (UniqueName: \"kubernetes.io/projected/ac58627b-09af-4b04-92ea-628bab81312a-kube-api-access-4cp8x\") pod \"cert-manager-cainjector-559cdd8c87-vqx7v\" (UID: \"ac58627b-09af-4b04-92ea-628bab81312a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" Jul 10 12:50:42.195474 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.195448 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac58627b-09af-4b04-92ea-628bab81312a-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vqx7v\" (UID: \"ac58627b-09af-4b04-92ea-628bab81312a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" Jul 10 12:50:42.195666 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.195646 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cp8x\" (UniqueName: \"kubernetes.io/projected/ac58627b-09af-4b04-92ea-628bab81312a-kube-api-access-4cp8x\") pod \"cert-manager-cainjector-559cdd8c87-vqx7v\" (UID: \"ac58627b-09af-4b04-92ea-628bab81312a\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" Jul 10 12:50:42.334478 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.334445 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" Jul 10 12:50:42.455530 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.455422 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v"] Jul 10 12:50:42.457667 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:50:42.457643 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac58627b_09af_4b04_92ea_628bab81312a.slice/crio-6c8ab5fe3c56a8d44a60c19a67fffff665de4f95623d40e428def83f57a2b032 WatchSource:0}: Error finding container 6c8ab5fe3c56a8d44a60c19a67fffff665de4f95623d40e428def83f57a2b032: Status 404 returned error can't find the container with id 6c8ab5fe3c56a8d44a60c19a67fffff665de4f95623d40e428def83f57a2b032 Jul 10 12:50:42.809763 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:42.809717 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" event={"ID":"ac58627b-09af-4b04-92ea-628bab81312a","Type":"ContainerStarted","Data":"6c8ab5fe3c56a8d44a60c19a67fffff665de4f95623d40e428def83f57a2b032"} Jul 10 12:50:44.822053 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:44.822016 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" event={"ID":"a699b834-5ad8-472b-b71b-c2217c4e00b1","Type":"ContainerStarted","Data":"009f0ab024752b202c3b643753d9b97cfd126609718890e5940bfe21865a63b8"} Jul 10 12:50:45.828136 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:45.828107 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" event={"ID":"ac58627b-09af-4b04-92ea-628bab81312a","Type":"ContainerStarted","Data":"c59d4e10c7185d68f4e9c7623ad1f65e332be5291338437314cac777fe0b25e3"} Jul 10 12:50:45.829880 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:45.829850 2571 generic.go:356] "Generic (PLEG): container finished" podID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerID="009f0ab024752b202c3b643753d9b97cfd126609718890e5940bfe21865a63b8" exitCode=0 Jul 10 12:50:45.830013 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:45.829925 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" event={"ID":"a699b834-5ad8-472b-b71b-c2217c4e00b1","Type":"ContainerDied","Data":"009f0ab024752b202c3b643753d9b97cfd126609718890e5940bfe21865a63b8"} Jul 10 12:50:45.845987 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:45.845950 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vqx7v" podStartSLOduration=2.221333517 podStartE2EDuration="4.845936219s" totalImagesPullingTime="2.624602702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:50:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:50:45.844812609 +0000 UTC m=+296.730573237" watchObservedRunningTime="2026-07-10 12:50:45.845936219 +0000 UTC m=+296.731696834" Jul 10 12:50:46.173514 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.173489 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tpv6b"] Jul 10 12:50:46.198209 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.198187 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tpv6b"] Jul 10 12:50:46.198348 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.198274 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:50:46.200987 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.200962 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7zzsl\"" Jul 10 12:50:46.227438 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.227414 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7437638a-f594-4346-8e79-76138483772c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tpv6b\" (UID: \"7437638a-f594-4346-8e79-76138483772c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:50:46.227540 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.227484 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67pxb\" (UniqueName: \"kubernetes.io/projected/7437638a-f594-4346-8e79-76138483772c-kube-api-access-67pxb\") pod \"cert-manager-webhook-6486ffd997-tpv6b\" (UID: \"7437638a-f594-4346-8e79-76138483772c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:50:46.328807 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.328784 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7437638a-f594-4346-8e79-76138483772c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tpv6b\" (UID: \"7437638a-f594-4346-8e79-76138483772c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:50:46.328909 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.328846 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67pxb\" (UniqueName: \"kubernetes.io/projected/7437638a-f594-4346-8e79-76138483772c-kube-api-access-67pxb\") pod \"cert-manager-webhook-6486ffd997-tpv6b\" (UID: \"7437638a-f594-4346-8e79-76138483772c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:50:46.337507 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.337486 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7437638a-f594-4346-8e79-76138483772c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tpv6b\" (UID: \"7437638a-f594-4346-8e79-76138483772c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:50:46.337843 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.337820 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67pxb\" (UniqueName: \"kubernetes.io/projected/7437638a-f594-4346-8e79-76138483772c-kube-api-access-67pxb\") pod \"cert-manager-webhook-6486ffd997-tpv6b\" (UID: \"7437638a-f594-4346-8e79-76138483772c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:50:46.506935 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.506874 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:50:46.652588 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.652567 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tpv6b"] Jul 10 12:50:46.654746 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:50:46.654721 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7437638a_f594_4346_8e79_76138483772c.slice/crio-538350dfe05afd8e1eacbe3ed5ca355b0813c10d442ee4c20b4471add0945543 WatchSource:0}: Error finding container 538350dfe05afd8e1eacbe3ed5ca355b0813c10d442ee4c20b4471add0945543: Status 404 returned error can't find the container with id 538350dfe05afd8e1eacbe3ed5ca355b0813c10d442ee4c20b4471add0945543 Jul 10 12:50:46.839994 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.839960 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" event={"ID":"7437638a-f594-4346-8e79-76138483772c","Type":"ContainerStarted","Data":"cc9f2ada307beda5a3672ade00b2747863d2fbe813596713b30b6238de9c91f0"} Jul 10 12:50:46.840448 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.840004 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" event={"ID":"7437638a-f594-4346-8e79-76138483772c","Type":"ContainerStarted","Data":"538350dfe05afd8e1eacbe3ed5ca355b0813c10d442ee4c20b4471add0945543"} Jul 10 12:50:46.840448 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.840118 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:50:46.841915 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.841889 2571 generic.go:356] "Generic (PLEG): container finished" podID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerID="be8297aaca8e57c5a9e9373a2a5bedbdaeaba07b02b3fd2ed3401cc84251f011" exitCode=0 Jul 10 12:50:46.842035 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.841965 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" event={"ID":"a699b834-5ad8-472b-b71b-c2217c4e00b1","Type":"ContainerDied","Data":"be8297aaca8e57c5a9e9373a2a5bedbdaeaba07b02b3fd2ed3401cc84251f011"} Jul 10 12:50:46.858861 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:46.858821 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" podStartSLOduration=0.858808483 podStartE2EDuration="858.808483ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:50:46.857071104 +0000 UTC m=+297.742831720" watchObservedRunningTime="2026-07-10 12:50:46.858808483 +0000 UTC m=+297.744569098" Jul 10 12:50:47.995849 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:47.995825 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:48.045040 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.045010 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-util\") pod \"a699b834-5ad8-472b-b71b-c2217c4e00b1\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " Jul 10 12:50:48.045172 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.045122 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ww5r\" (UniqueName: \"kubernetes.io/projected/a699b834-5ad8-472b-b71b-c2217c4e00b1-kube-api-access-2ww5r\") pod \"a699b834-5ad8-472b-b71b-c2217c4e00b1\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " Jul 10 12:50:48.045246 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.045180 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-bundle\") pod \"a699b834-5ad8-472b-b71b-c2217c4e00b1\" (UID: \"a699b834-5ad8-472b-b71b-c2217c4e00b1\") " Jul 10 12:50:48.045555 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.045526 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-bundle" (OuterVolumeSpecName: "bundle") pod "a699b834-5ad8-472b-b71b-c2217c4e00b1" (UID: "a699b834-5ad8-472b-b71b-c2217c4e00b1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:50:48.047088 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.047063 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a699b834-5ad8-472b-b71b-c2217c4e00b1-kube-api-access-2ww5r" (OuterVolumeSpecName: "kube-api-access-2ww5r") pod "a699b834-5ad8-472b-b71b-c2217c4e00b1" (UID: "a699b834-5ad8-472b-b71b-c2217c4e00b1"). InnerVolumeSpecName "kube-api-access-2ww5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:50:48.049110 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.049088 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-util" (OuterVolumeSpecName: "util") pod "a699b834-5ad8-472b-b71b-c2217c4e00b1" (UID: "a699b834-5ad8-472b-b71b-c2217c4e00b1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:50:48.145789 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.145763 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:50:48.145789 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.145786 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a699b834-5ad8-472b-b71b-c2217c4e00b1-util\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:50:48.145978 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.145795 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ww5r\" (UniqueName: \"kubernetes.io/projected/a699b834-5ad8-472b-b71b-c2217c4e00b1-kube-api-access-2ww5r\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:50:48.852169 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.852144 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" Jul 10 12:50:48.852403 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.852175 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjmvck" event={"ID":"a699b834-5ad8-472b-b71b-c2217c4e00b1","Type":"ContainerDied","Data":"84e7942e47479bdfd7d69d6e86e651d9ae48bc9656955b4448725b625dc76367"} Jul 10 12:50:48.852403 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:48.852220 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84e7942e47479bdfd7d69d6e86e651d9ae48bc9656955b4448725b625dc76367" Jul 10 12:50:49.466926 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:49.466890 2571 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 12:50:52.848727 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:50:52.848697 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tpv6b" Jul 10 12:51:00.310423 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.310391 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sms29"] Jul 10 12:51:00.310823 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.310810 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerName="util" Jul 10 12:51:00.310823 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.310823 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerName="util" Jul 10 12:51:00.310898 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.310844 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerName="pull" Jul 10 12:51:00.310898 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.310850 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerName="pull" Jul 10 12:51:00.310898 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.310862 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerName="extract" Jul 10 12:51:00.310898 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.310871 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerName="extract" Jul 10 12:51:00.311017 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.310937 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a699b834-5ad8-472b-b71b-c2217c4e00b1" containerName="extract" Jul 10 12:51:00.331194 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.331173 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sms29" Jul 10 12:51:00.331194 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.331174 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sms29"] Jul 10 12:51:00.333743 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.333720 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5rfhx\"" Jul 10 12:51:00.438477 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.438454 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/615cd833-f974-4d35-a00e-cf9a07e2e7f7-bound-sa-token\") pod \"cert-manager-55b4d954d5-sms29\" (UID: \"615cd833-f974-4d35-a00e-cf9a07e2e7f7\") " pod="cert-manager/cert-manager-55b4d954d5-sms29" Jul 10 12:51:00.438612 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.438509 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-725lt\" (UniqueName: \"kubernetes.io/projected/615cd833-f974-4d35-a00e-cf9a07e2e7f7-kube-api-access-725lt\") pod \"cert-manager-55b4d954d5-sms29\" (UID: \"615cd833-f974-4d35-a00e-cf9a07e2e7f7\") " pod="cert-manager/cert-manager-55b4d954d5-sms29" Jul 10 12:51:00.538845 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.538822 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/615cd833-f974-4d35-a00e-cf9a07e2e7f7-bound-sa-token\") pod \"cert-manager-55b4d954d5-sms29\" (UID: \"615cd833-f974-4d35-a00e-cf9a07e2e7f7\") " pod="cert-manager/cert-manager-55b4d954d5-sms29" Jul 10 12:51:00.538950 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.538870 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-725lt\" (UniqueName: \"kubernetes.io/projected/615cd833-f974-4d35-a00e-cf9a07e2e7f7-kube-api-access-725lt\") pod \"cert-manager-55b4d954d5-sms29\" (UID: \"615cd833-f974-4d35-a00e-cf9a07e2e7f7\") " pod="cert-manager/cert-manager-55b4d954d5-sms29" Jul 10 12:51:00.547036 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.547015 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-725lt\" (UniqueName: \"kubernetes.io/projected/615cd833-f974-4d35-a00e-cf9a07e2e7f7-kube-api-access-725lt\") pod \"cert-manager-55b4d954d5-sms29\" (UID: \"615cd833-f974-4d35-a00e-cf9a07e2e7f7\") " pod="cert-manager/cert-manager-55b4d954d5-sms29" Jul 10 12:51:00.547122 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.547087 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/615cd833-f974-4d35-a00e-cf9a07e2e7f7-bound-sa-token\") pod \"cert-manager-55b4d954d5-sms29\" (UID: \"615cd833-f974-4d35-a00e-cf9a07e2e7f7\") " pod="cert-manager/cert-manager-55b4d954d5-sms29" Jul 10 12:51:00.639848 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.639761 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sms29" Jul 10 12:51:00.780066 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.780017 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sms29"] Jul 10 12:51:00.782258 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:00.782225 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod615cd833_f974_4d35_a00e_cf9a07e2e7f7.slice/crio-7084dc6fa731aa1048aa2dcf3f692968c3a678611429abeb58fb9f9430e64230 WatchSource:0}: Error finding container 7084dc6fa731aa1048aa2dcf3f692968c3a678611429abeb58fb9f9430e64230: Status 404 returned error can't find the container with id 7084dc6fa731aa1048aa2dcf3f692968c3a678611429abeb58fb9f9430e64230 Jul 10 12:51:00.905735 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.905706 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sms29" event={"ID":"615cd833-f974-4d35-a00e-cf9a07e2e7f7","Type":"ContainerStarted","Data":"cf97c58f533ac56da7f5d1a78a7c8ee9b983a5d45c585d78c0b8b474783f1d7a"} Jul 10 12:51:00.905850 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.905753 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sms29" event={"ID":"615cd833-f974-4d35-a00e-cf9a07e2e7f7","Type":"ContainerStarted","Data":"7084dc6fa731aa1048aa2dcf3f692968c3a678611429abeb58fb9f9430e64230"} Jul 10 12:51:00.923485 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:00.923443 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-sms29" podStartSLOduration=0.923428134 podStartE2EDuration="923.428134ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:51:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:51:00.921678966 +0000 UTC m=+311.807439580" watchObservedRunningTime="2026-07-10 12:51:00.923428134 +0000 UTC m=+311.809188749" Jul 10 12:51:06.161608 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.161576 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s"] Jul 10 12:51:06.164107 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.164091 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.166598 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.166578 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 12:51:06.166598 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.166593 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8p5fv\"" Jul 10 12:51:06.166772 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.166667 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 12:51:06.172634 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.172611 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s"] Jul 10 12:51:06.286490 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.286465 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.286630 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.286504 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.286630 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.286595 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l6j5\" (UniqueName: \"kubernetes.io/projected/c2d6b047-f153-4561-9d7f-7891f740628b-kube-api-access-8l6j5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.387402 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.387371 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.387537 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.387419 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.387537 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.387488 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l6j5\" (UniqueName: \"kubernetes.io/projected/c2d6b047-f153-4561-9d7f-7891f740628b-kube-api-access-8l6j5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.387720 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.387700 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.387788 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.387759 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.396060 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.396041 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l6j5\" (UniqueName: \"kubernetes.io/projected/c2d6b047-f153-4561-9d7f-7891f740628b-kube-api-access-8l6j5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.474031 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.473976 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:06.605806 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.603076 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s"] Jul 10 12:51:06.608034 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:06.607996 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2d6b047_f153_4561_9d7f_7891f740628b.slice/crio-f1fa32856363bf11630412c55dd7a6fa57e5d1fa625815846783de2c462ae3af WatchSource:0}: Error finding container f1fa32856363bf11630412c55dd7a6fa57e5d1fa625815846783de2c462ae3af: Status 404 returned error can't find the container with id f1fa32856363bf11630412c55dd7a6fa57e5d1fa625815846783de2c462ae3af Jul 10 12:51:06.932080 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.932050 2571 generic.go:356] "Generic (PLEG): container finished" podID="c2d6b047-f153-4561-9d7f-7891f740628b" containerID="e8b988d65416db4ed2c6058aa485958db7d5940bdefc7bcbaf3a73ba1f585076" exitCode=0 Jul 10 12:51:06.932210 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.932137 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" event={"ID":"c2d6b047-f153-4561-9d7f-7891f740628b","Type":"ContainerDied","Data":"e8b988d65416db4ed2c6058aa485958db7d5940bdefc7bcbaf3a73ba1f585076"} Jul 10 12:51:06.932210 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.932183 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" event={"ID":"c2d6b047-f153-4561-9d7f-7891f740628b","Type":"ContainerStarted","Data":"f1fa32856363bf11630412c55dd7a6fa57e5d1fa625815846783de2c462ae3af"} Jul 10 12:51:06.932598 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:06.932583 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 12:51:07.938518 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:07.938435 2571 generic.go:356] "Generic (PLEG): container finished" podID="c2d6b047-f153-4561-9d7f-7891f740628b" containerID="92e7e87962d3f43093bf0cb99864d394d9ce280a88b7515ac64cc1eb5e05adab" exitCode=0 Jul 10 12:51:07.938518 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:07.938485 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" event={"ID":"c2d6b047-f153-4561-9d7f-7891f740628b","Type":"ContainerDied","Data":"92e7e87962d3f43093bf0cb99864d394d9ce280a88b7515ac64cc1eb5e05adab"} Jul 10 12:51:08.945521 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:08.945489 2571 generic.go:356] "Generic (PLEG): container finished" podID="c2d6b047-f153-4561-9d7f-7891f740628b" containerID="31616dd029efce7025d8b58874bde70d9764c1d6429e48e2d6d24771e416761e" exitCode=0 Jul 10 12:51:08.945916 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:08.945550 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" event={"ID":"c2d6b047-f153-4561-9d7f-7891f740628b","Type":"ContainerDied","Data":"31616dd029efce7025d8b58874bde70d9764c1d6429e48e2d6d24771e416761e"} Jul 10 12:51:10.078063 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.078042 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:10.227097 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.227017 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l6j5\" (UniqueName: \"kubernetes.io/projected/c2d6b047-f153-4561-9d7f-7891f740628b-kube-api-access-8l6j5\") pod \"c2d6b047-f153-4561-9d7f-7891f740628b\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " Jul 10 12:51:10.227097 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.227094 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-bundle\") pod \"c2d6b047-f153-4561-9d7f-7891f740628b\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " Jul 10 12:51:10.227275 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.227128 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-util\") pod \"c2d6b047-f153-4561-9d7f-7891f740628b\" (UID: \"c2d6b047-f153-4561-9d7f-7891f740628b\") " Jul 10 12:51:10.227788 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.227758 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-bundle" (OuterVolumeSpecName: "bundle") pod "c2d6b047-f153-4561-9d7f-7891f740628b" (UID: "c2d6b047-f153-4561-9d7f-7891f740628b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:51:10.229074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.229043 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2d6b047-f153-4561-9d7f-7891f740628b-kube-api-access-8l6j5" (OuterVolumeSpecName: "kube-api-access-8l6j5") pod "c2d6b047-f153-4561-9d7f-7891f740628b" (UID: "c2d6b047-f153-4561-9d7f-7891f740628b"). InnerVolumeSpecName "kube-api-access-8l6j5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:51:10.232737 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.232711 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-util" (OuterVolumeSpecName: "util") pod "c2d6b047-f153-4561-9d7f-7891f740628b" (UID: "c2d6b047-f153-4561-9d7f-7891f740628b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:51:10.327779 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.327752 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-util\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:10.327779 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.327777 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l6j5\" (UniqueName: \"kubernetes.io/projected/c2d6b047-f153-4561-9d7f-7891f740628b-kube-api-access-8l6j5\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:10.327946 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.327787 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c2d6b047-f153-4561-9d7f-7891f740628b-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:10.956488 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.956455 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" event={"ID":"c2d6b047-f153-4561-9d7f-7891f740628b","Type":"ContainerDied","Data":"f1fa32856363bf11630412c55dd7a6fa57e5d1fa625815846783de2c462ae3af"} Jul 10 12:51:10.956647 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.956494 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkl2s" Jul 10 12:51:10.956647 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:10.956501 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1fa32856363bf11630412c55dd7a6fa57e5d1fa625815846783de2c462ae3af" Jul 10 12:51:17.735923 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.735885 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl"] Jul 10 12:51:17.736263 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.736239 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d6b047-f153-4561-9d7f-7891f740628b" containerName="extract" Jul 10 12:51:17.736263 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.736250 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d6b047-f153-4561-9d7f-7891f740628b" containerName="extract" Jul 10 12:51:17.736263 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.736261 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d6b047-f153-4561-9d7f-7891f740628b" containerName="pull" Jul 10 12:51:17.736389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.736266 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d6b047-f153-4561-9d7f-7891f740628b" containerName="pull" Jul 10 12:51:17.736389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.736274 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d6b047-f153-4561-9d7f-7891f740628b" containerName="util" Jul 10 12:51:17.736389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.736280 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d6b047-f153-4561-9d7f-7891f740628b" containerName="util" Jul 10 12:51:17.736389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.736358 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2d6b047-f153-4561-9d7f-7891f740628b" containerName="extract" Jul 10 12:51:17.740504 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.740487 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.743211 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.743195 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 12:51:17.743543 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.743528 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 12:51:17.743694 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.743630 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 12:51:17.744366 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.744337 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 12:51:17.744481 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.744466 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6rhzd\"" Jul 10 12:51:17.754682 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.754661 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl"] Jul 10 12:51:17.885137 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.885097 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85c83e63-b8ee-4e12-ab12-de17359db348-manifests\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.885283 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.885157 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85c83e63-b8ee-4e12-ab12-de17359db348-tmp\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.885283 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.885188 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlcb6\" (UniqueName: \"kubernetes.io/projected/85c83e63-b8ee-4e12-ab12-de17359db348-kube-api-access-qlcb6\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.885283 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.885211 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85c83e63-b8ee-4e12-ab12-de17359db348-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.885425 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.885285 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85c83e63-b8ee-4e12-ab12-de17359db348-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.985725 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.985679 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85c83e63-b8ee-4e12-ab12-de17359db348-manifests\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.985851 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.985778 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85c83e63-b8ee-4e12-ab12-de17359db348-tmp\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.985851 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.985828 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlcb6\" (UniqueName: \"kubernetes.io/projected/85c83e63-b8ee-4e12-ab12-de17359db348-kube-api-access-qlcb6\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.985965 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.985859 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85c83e63-b8ee-4e12-ab12-de17359db348-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.985965 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.985915 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85c83e63-b8ee-4e12-ab12-de17359db348-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.986082 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.986050 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85c83e63-b8ee-4e12-ab12-de17359db348-manifests\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.987940 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.987914 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85c83e63-b8ee-4e12-ab12-de17359db348-tmp\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.988214 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.988193 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85c83e63-b8ee-4e12-ab12-de17359db348-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.988581 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.988562 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85c83e63-b8ee-4e12-ab12-de17359db348-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:17.997913 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:17.997891 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlcb6\" (UniqueName: \"kubernetes.io/projected/85c83e63-b8ee-4e12-ab12-de17359db348-kube-api-access-qlcb6\") pod \"opendatahub-operator-controller-manager-7f6798dbc-44fgl\" (UID: \"85c83e63-b8ee-4e12-ab12-de17359db348\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:18.050652 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:18.050620 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:18.213417 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:18.213391 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl"] Jul 10 12:51:18.214915 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:18.214885 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c83e63_b8ee_4e12_ab12_de17359db348.slice/crio-7a695faec4f58b871f4130820066406f4300bc397c074b17a41bf488e311d0d1 WatchSource:0}: Error finding container 7a695faec4f58b871f4130820066406f4300bc397c074b17a41bf488e311d0d1: Status 404 returned error can't find the container with id 7a695faec4f58b871f4130820066406f4300bc397c074b17a41bf488e311d0d1 Jul 10 12:51:18.998659 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:18.998621 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" event={"ID":"85c83e63-b8ee-4e12-ab12-de17359db348","Type":"ContainerStarted","Data":"7a695faec4f58b871f4130820066406f4300bc397c074b17a41bf488e311d0d1"} Jul 10 12:51:22.018003 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:22.017970 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" event={"ID":"85c83e63-b8ee-4e12-ab12-de17359db348","Type":"ContainerStarted","Data":"395a6fbeecf1b54a96a3d6af7e2ae5762fd8c116bdcb247a385fc557b9850f5c"} Jul 10 12:51:23.024021 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:23.023983 2571 generic.go:356] "Generic (PLEG): container finished" podID="85c83e63-b8ee-4e12-ab12-de17359db348" containerID="395a6fbeecf1b54a96a3d6af7e2ae5762fd8c116bdcb247a385fc557b9850f5c" exitCode=0 Jul 10 12:51:23.024473 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:23.024069 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" event={"ID":"85c83e63-b8ee-4e12-ab12-de17359db348","Type":"ContainerDied","Data":"395a6fbeecf1b54a96a3d6af7e2ae5762fd8c116bdcb247a385fc557b9850f5c"} Jul 10 12:51:24.030703 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:24.030667 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" event={"ID":"85c83e63-b8ee-4e12-ab12-de17359db348","Type":"ContainerStarted","Data":"b8f78807b7364f611ad52499e4061ddf497b8ad980f4350fea2071aa52d101af"} Jul 10 12:51:24.031106 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:24.030775 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:24.058557 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:24.056912 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" podStartSLOduration=2.591427666 podStartE2EDuration="7.056893655s" totalImagesPullingTime="3.404853092s" totalInitContainerRuntime="1.060612897s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:51:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:51:24.054534225 +0000 UTC m=+334.940294840" watchObservedRunningTime="2026-07-10 12:51:24.056893655 +0000 UTC m=+334.942654270" Jul 10 12:51:35.037383 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:35.037344 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f6798dbc-44fgl" Jul 10 12:51:37.494488 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.494451 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n"] Jul 10 12:51:37.503103 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.503079 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:37.507138 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.507101 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 12:51:37.507944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.507927 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tsxbq\"" Jul 10 12:51:37.509486 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.509465 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 12:51:37.511543 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.511523 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n"] Jul 10 12:51:37.537671 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.537644 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d111e575-84e5-4a66-9401-c0beefbb3176-tls-certs\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:37.537814 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.537777 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d111e575-84e5-4a66-9401-c0beefbb3176-tmp\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:37.537814 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.537804 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwptg\" (UniqueName: \"kubernetes.io/projected/d111e575-84e5-4a66-9401-c0beefbb3176-kube-api-access-cwptg\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:37.638867 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.638818 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d111e575-84e5-4a66-9401-c0beefbb3176-tls-certs\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:37.639050 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.638942 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d111e575-84e5-4a66-9401-c0beefbb3176-tmp\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:37.639050 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.638967 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwptg\" (UniqueName: \"kubernetes.io/projected/d111e575-84e5-4a66-9401-c0beefbb3176-kube-api-access-cwptg\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:37.639050 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:51:37.638979 2571 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 10 12:51:37.639183 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:51:37.639052 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d111e575-84e5-4a66-9401-c0beefbb3176-tls-certs podName:d111e575-84e5-4a66-9401-c0beefbb3176 nodeName:}" failed. No retries permitted until 2026-07-10 12:51:38.139030717 +0000 UTC m=+349.024791328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d111e575-84e5-4a66-9401-c0beefbb3176-tls-certs") pod "kube-auth-proxy-779ffb979b-xnq4n" (UID: "d111e575-84e5-4a66-9401-c0beefbb3176") : secret "kube-auth-proxy-tls" not found Jul 10 12:51:37.641177 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.641152 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d111e575-84e5-4a66-9401-c0beefbb3176-tmp\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:37.664822 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.664802 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwptg\" (UniqueName: \"kubernetes.io/projected/d111e575-84e5-4a66-9401-c0beefbb3176-kube-api-access-cwptg\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:37.706780 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.706755 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5q4nh"] Jul 10 12:51:37.710189 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.710165 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:37.714512 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.714491 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-shphd\"" Jul 10 12:51:37.721664 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.721647 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 12:51:37.731914 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.731893 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5q4nh"] Jul 10 12:51:37.740353 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.740328 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eae18245-0cf2-4dac-af9e-1c1a7972d7e0-cert\") pod \"odh-model-controller-7998c67dc4-5q4nh\" (UID: \"eae18245-0cf2-4dac-af9e-1c1a7972d7e0\") " pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:37.740513 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.740492 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9dmb\" (UniqueName: \"kubernetes.io/projected/eae18245-0cf2-4dac-af9e-1c1a7972d7e0-kube-api-access-f9dmb\") pod \"odh-model-controller-7998c67dc4-5q4nh\" (UID: \"eae18245-0cf2-4dac-af9e-1c1a7972d7e0\") " pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:37.841673 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.841603 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9dmb\" (UniqueName: \"kubernetes.io/projected/eae18245-0cf2-4dac-af9e-1c1a7972d7e0-kube-api-access-f9dmb\") pod \"odh-model-controller-7998c67dc4-5q4nh\" (UID: \"eae18245-0cf2-4dac-af9e-1c1a7972d7e0\") " pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:37.841673 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.841653 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eae18245-0cf2-4dac-af9e-1c1a7972d7e0-cert\") pod \"odh-model-controller-7998c67dc4-5q4nh\" (UID: \"eae18245-0cf2-4dac-af9e-1c1a7972d7e0\") " pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:37.841829 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:51:37.841779 2571 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 10 12:51:37.841829 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:51:37.841825 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eae18245-0cf2-4dac-af9e-1c1a7972d7e0-cert podName:eae18245-0cf2-4dac-af9e-1c1a7972d7e0 nodeName:}" failed. No retries permitted until 2026-07-10 12:51:38.341812776 +0000 UTC m=+349.227573370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/eae18245-0cf2-4dac-af9e-1c1a7972d7e0-cert") pod "odh-model-controller-7998c67dc4-5q4nh" (UID: "eae18245-0cf2-4dac-af9e-1c1a7972d7e0") : secret "odh-model-controller-webhook-cert" not found Jul 10 12:51:37.850614 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:37.850587 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9dmb\" (UniqueName: \"kubernetes.io/projected/eae18245-0cf2-4dac-af9e-1c1a7972d7e0-kube-api-access-f9dmb\") pod \"odh-model-controller-7998c67dc4-5q4nh\" (UID: \"eae18245-0cf2-4dac-af9e-1c1a7972d7e0\") " pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:38.144201 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.144168 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d111e575-84e5-4a66-9401-c0beefbb3176-tls-certs\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:38.154833 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.154796 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d111e575-84e5-4a66-9401-c0beefbb3176-tls-certs\") pod \"kube-auth-proxy-779ffb979b-xnq4n\" (UID: \"d111e575-84e5-4a66-9401-c0beefbb3176\") " pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:38.346521 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.346487 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eae18245-0cf2-4dac-af9e-1c1a7972d7e0-cert\") pod \"odh-model-controller-7998c67dc4-5q4nh\" (UID: \"eae18245-0cf2-4dac-af9e-1c1a7972d7e0\") " pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:38.348876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.348857 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eae18245-0cf2-4dac-af9e-1c1a7972d7e0-cert\") pod \"odh-model-controller-7998c67dc4-5q4nh\" (UID: \"eae18245-0cf2-4dac-af9e-1c1a7972d7e0\") " pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:38.414404 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.414313 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" Jul 10 12:51:38.554427 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.554361 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n"] Jul 10 12:51:38.556694 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:38.556673 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd111e575_84e5_4a66_9401_c0beefbb3176.slice/crio-74e734e8caab9ff7fc80140241ed3832c6c78a406b89372492a2c0bb841c0dd2 WatchSource:0}: Error finding container 74e734e8caab9ff7fc80140241ed3832c6c78a406b89372492a2c0bb841c0dd2: Status 404 returned error can't find the container with id 74e734e8caab9ff7fc80140241ed3832c6c78a406b89372492a2c0bb841c0dd2 Jul 10 12:51:38.620903 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.620878 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:38.700206 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.700181 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dfc9h"] Jul 10 12:51:38.704792 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.704773 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:38.707582 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.707384 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 10 12:51:38.721196 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.721171 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dfc9h"] Jul 10 12:51:38.749430 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.749402 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36c309c1-45e5-479f-93b2-51fd699e86f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-dfc9h\" (UID: \"36c309c1-45e5-479f-93b2-51fd699e86f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:38.749564 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.749457 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dmrl\" (UniqueName: \"kubernetes.io/projected/36c309c1-45e5-479f-93b2-51fd699e86f2-kube-api-access-2dmrl\") pod \"model-serving-api-54d89bd5fc-dfc9h\" (UID: \"36c309c1-45e5-479f-93b2-51fd699e86f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:38.767110 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.767085 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5q4nh"] Jul 10 12:51:38.769851 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:38.769829 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae18245_0cf2_4dac_af9e_1c1a7972d7e0.slice/crio-3b6cbc5e5746488fed6d35ab36041f2edea97293756c4b98d9ec34788107d07a WatchSource:0}: Error finding container 3b6cbc5e5746488fed6d35ab36041f2edea97293756c4b98d9ec34788107d07a: Status 404 returned error can't find the container with id 3b6cbc5e5746488fed6d35ab36041f2edea97293756c4b98d9ec34788107d07a Jul 10 12:51:38.849864 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.849842 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36c309c1-45e5-479f-93b2-51fd699e86f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-dfc9h\" (UID: \"36c309c1-45e5-479f-93b2-51fd699e86f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:38.849997 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.849886 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dmrl\" (UniqueName: \"kubernetes.io/projected/36c309c1-45e5-479f-93b2-51fd699e86f2-kube-api-access-2dmrl\") pod \"model-serving-api-54d89bd5fc-dfc9h\" (UID: \"36c309c1-45e5-479f-93b2-51fd699e86f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:38.850062 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:51:38.850006 2571 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 10 12:51:38.850114 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:51:38.850076 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36c309c1-45e5-479f-93b2-51fd699e86f2-tls-certs podName:36c309c1-45e5-479f-93b2-51fd699e86f2 nodeName:}" failed. No retries permitted until 2026-07-10 12:51:39.350056106 +0000 UTC m=+350.235816713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/36c309c1-45e5-479f-93b2-51fd699e86f2-tls-certs") pod "model-serving-api-54d89bd5fc-dfc9h" (UID: "36c309c1-45e5-479f-93b2-51fd699e86f2") : secret "model-serving-api-tls" not found Jul 10 12:51:38.860058 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.860032 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dmrl\" (UniqueName: \"kubernetes.io/projected/36c309c1-45e5-479f-93b2-51fd699e86f2-kube-api-access-2dmrl\") pod \"model-serving-api-54d89bd5fc-dfc9h\" (UID: \"36c309c1-45e5-479f-93b2-51fd699e86f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:38.930613 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.930538 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7"] Jul 10 12:51:38.933989 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.933972 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:38.936269 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.936252 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 12:51:38.936380 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.936286 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 12:51:38.936380 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.936354 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jrrdz\"" Jul 10 12:51:38.936380 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.936306 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 12:51:38.936547 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.936362 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 12:51:38.951214 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.951194 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:38.951309 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.951262 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhqzx\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-kube-api-access-lhqzx\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:38.951355 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.951282 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/16602aec-ffe6-4507-a782-4c7edd04f6d1-local-certs\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:38.951355 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.951327 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-token\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:38.951355 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.951351 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:38.951455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.951372 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-cacerts\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:38.951455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.951435 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:38.955912 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:38.955892 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7"] Jul 10 12:51:39.052369 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.052341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhqzx\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-kube-api-access-lhqzx\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.052504 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.052374 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/16602aec-ffe6-4507-a782-4c7edd04f6d1-local-certs\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.052504 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.052391 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-token\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.052504 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.052415 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.052504 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.052436 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-cacerts\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.052504 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.052455 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.052754 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.052573 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.053176 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.053149 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.054715 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.054688 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.054715 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.054692 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/16602aec-ffe6-4507-a782-4c7edd04f6d1-local-certs\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.054884 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.054804 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-cacerts\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.054884 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.054840 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.067035 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.067009 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-token\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.068079 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.068048 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhqzx\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-kube-api-access-lhqzx\") pod \"istiod-openshift-gateway-7574459656-zdrf7\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.102245 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.102216 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" event={"ID":"d111e575-84e5-4a66-9401-c0beefbb3176","Type":"ContainerStarted","Data":"74e734e8caab9ff7fc80140241ed3832c6c78a406b89372492a2c0bb841c0dd2"} Jul 10 12:51:39.103350 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.103328 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" event={"ID":"eae18245-0cf2-4dac-af9e-1c1a7972d7e0","Type":"ContainerStarted","Data":"3b6cbc5e5746488fed6d35ab36041f2edea97293756c4b98d9ec34788107d07a"} Jul 10 12:51:39.243971 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.243884 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:39.355207 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.355172 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36c309c1-45e5-479f-93b2-51fd699e86f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-dfc9h\" (UID: \"36c309c1-45e5-479f-93b2-51fd699e86f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:39.357838 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.357817 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36c309c1-45e5-479f-93b2-51fd699e86f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-dfc9h\" (UID: \"36c309c1-45e5-479f-93b2-51fd699e86f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:39.388443 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.388412 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7"] Jul 10 12:51:39.390421 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:39.390392 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16602aec_ffe6_4507_a782_4c7edd04f6d1.slice/crio-35ce50058dba519f690495d0fa8af3e3e45b7d047d8f8bb50e66c0308148a203 WatchSource:0}: Error finding container 35ce50058dba519f690495d0fa8af3e3e45b7d047d8f8bb50e66c0308148a203: Status 404 returned error can't find the container with id 35ce50058dba519f690495d0fa8af3e3e45b7d047d8f8bb50e66c0308148a203 Jul 10 12:51:39.616832 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.616758 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:39.792540 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:39.792410 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dfc9h"] Jul 10 12:51:39.799508 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:39.799470 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c309c1_45e5_479f_93b2_51fd699e86f2.slice/crio-6141affd39d4d8f245c8daa571e4344df8868cfae92ed2997fee58e3b3252a2e WatchSource:0}: Error finding container 6141affd39d4d8f245c8daa571e4344df8868cfae92ed2997fee58e3b3252a2e: Status 404 returned error can't find the container with id 6141affd39d4d8f245c8daa571e4344df8868cfae92ed2997fee58e3b3252a2e Jul 10 12:51:40.113752 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:40.113660 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" event={"ID":"36c309c1-45e5-479f-93b2-51fd699e86f2","Type":"ContainerStarted","Data":"6141affd39d4d8f245c8daa571e4344df8868cfae92ed2997fee58e3b3252a2e"} Jul 10 12:51:40.115814 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:40.115783 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" event={"ID":"16602aec-ffe6-4507-a782-4c7edd04f6d1","Type":"ContainerStarted","Data":"35ce50058dba519f690495d0fa8af3e3e45b7d047d8f8bb50e66c0308148a203"} Jul 10 12:51:43.948199 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:43.948159 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:51:43.948537 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:43.948251 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:51:44.143529 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.143495 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" event={"ID":"d111e575-84e5-4a66-9401-c0beefbb3176","Type":"ContainerStarted","Data":"edc43111bd8cb85ce0b67fd6b15ee71390d0959ef3a2c6bbd7436a16b92a8712"} Jul 10 12:51:44.145191 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.145166 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" event={"ID":"eae18245-0cf2-4dac-af9e-1c1a7972d7e0","Type":"ContainerStarted","Data":"dabb3ef544289ff8f161953d93179dad79cad877c9b9fad72d5872f6c0bcff66"} Jul 10 12:51:44.145354 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.145237 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:44.147309 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.147261 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" event={"ID":"16602aec-ffe6-4507-a782-4c7edd04f6d1","Type":"ContainerStarted","Data":"48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb"} Jul 10 12:51:44.147438 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.147334 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:44.148768 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.148750 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" event={"ID":"36c309c1-45e5-479f-93b2-51fd699e86f2","Type":"ContainerStarted","Data":"87ff27be54eb2d62bc9b8784aa032112539503ba2f0a000c498bdbdd6a198be3"} Jul 10 12:51:44.148855 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.148803 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:51:44.161414 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.161331 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-779ffb979b-xnq4n" podStartSLOduration=1.777404008 podStartE2EDuration="7.161317778s" totalImagesPullingTime="5.38391377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:51:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:51:44.159359426 +0000 UTC m=+355.045120042" watchObservedRunningTime="2026-07-10 12:51:44.161317778 +0000 UTC m=+355.047078394" Jul 10 12:51:44.181013 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.180959 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" podStartSLOduration=1.625870832 podStartE2EDuration="6.180941953s" totalImagesPullingTime="4.555071121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:51:44.178151134 +0000 UTC m=+355.063911750" watchObservedRunningTime="2026-07-10 12:51:44.180941953 +0000 UTC m=+355.066702568" Jul 10 12:51:44.197203 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.197160 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" podStartSLOduration=2.056857745 podStartE2EDuration="6.197149195s" totalImagesPullingTime="4.14029145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:51:44.196417689 +0000 UTC m=+355.082178305" watchObservedRunningTime="2026-07-10 12:51:44.197149195 +0000 UTC m=+355.082909809" Jul 10 12:51:44.212625 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:44.212582 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" podStartSLOduration=2.04150817 podStartE2EDuration="7.212564424s" totalImagesPullingTime="5.171056254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:51:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:51:44.210704995 +0000 UTC m=+355.096465612" watchObservedRunningTime="2026-07-10 12:51:44.212564424 +0000 UTC m=+355.098325040" Jul 10 12:51:45.155541 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.155501 2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-zdrf7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 12:51:45.155901 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.155554 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" podUID="16602aec-ffe6-4507-a782-4c7edd04f6d1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 12:51:45.642145 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.642119 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g"] Jul 10 12:51:45.645842 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.645800 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.650460 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.650029 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pd5rp\"" Jul 10 12:51:45.683746 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.683719 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g"] Jul 10 12:51:45.716639 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.716605 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.716639 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.716642 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.716811 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.716662 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-data\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.716811 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.716765 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.716881 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.716817 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.716881 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.716838 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-token\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.716881 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.716863 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.716972 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.716887 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxvsp\" (UniqueName: \"kubernetes.io/projected/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-kube-api-access-mxvsp\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.716972 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.716922 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.817859 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.817824 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818014 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.817910 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818014 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.817935 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818014 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.817954 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-data\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818014 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.817993 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818207 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.818028 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818207 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.818048 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-token\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818207 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.818075 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818207 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.818092 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxvsp\" (UniqueName: \"kubernetes.io/projected/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-kube-api-access-mxvsp\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818416 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.818340 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818475 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.818437 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-data\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818603 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.818492 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818603 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.818553 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.818807 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.818755 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.820462 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.820431 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.820710 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.820694 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.828336 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.828309 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxvsp\" (UniqueName: \"kubernetes.io/projected/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-kube-api-access-mxvsp\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.828603 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.828583 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbd98ae2-6a89-4ad5-8614-96ea5f071f72-istio-token\") pod \"data-science-gateway-data-science-gateway-class-695cd9447dsp57g\" (UID: \"fbd98ae2-6a89-4ad5-8614-96ea5f071f72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:45.964167 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:45.964060 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:46.117968 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:46.117795 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g"] Jul 10 12:51:46.120578 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:46.120546 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd98ae2_6a89_4ad5_8614_96ea5f071f72.slice/crio-2ecec2b0a280a99cff3c800f9c09df5cbc27de16998bac735030af48c0ce0582 WatchSource:0}: Error finding container 2ecec2b0a280a99cff3c800f9c09df5cbc27de16998bac735030af48c0ce0582: Status 404 returned error can't find the container with id 2ecec2b0a280a99cff3c800f9c09df5cbc27de16998bac735030af48c0ce0582 Jul 10 12:51:46.159908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:46.159879 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" event={"ID":"fbd98ae2-6a89-4ad5-8614-96ea5f071f72","Type":"ContainerStarted","Data":"2ecec2b0a280a99cff3c800f9c09df5cbc27de16998bac735030af48c0ce0582"} Jul 10 12:51:48.102319 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.102275 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj"] Jul 10 12:51:48.109334 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.109280 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:48.112084 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.112059 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 12:51:48.112212 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.112119 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qspxg\"" Jul 10 12:51:48.123734 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.123715 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj"] Jul 10 12:51:48.128847 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.128827 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc"] Jul 10 12:51:48.132438 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.132419 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:51:48.135064 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.135045 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 10 12:51:48.135181 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.135109 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-qxq8m\"" Jul 10 12:51:48.148108 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.148089 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc"] Jul 10 12:51:48.155613 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.155556 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:48.238644 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.238548 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67976ed9-c2b3-4983-8196-f22ec66af6bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-lsdvj\" (UID: \"67976ed9-c2b3-4983-8196-f22ec66af6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:48.238644 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.238636 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc0227f-b22b-4207-828a-415efd0d2b9d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xcvtc\" (UID: \"8dc0227f-b22b-4207-828a-415efd0d2b9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:51:48.238849 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.238688 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86rg7\" (UniqueName: \"kubernetes.io/projected/67976ed9-c2b3-4983-8196-f22ec66af6bd-kube-api-access-86rg7\") pod \"kserve-controller-manager-d6f4b65b8-lsdvj\" (UID: \"67976ed9-c2b3-4983-8196-f22ec66af6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:48.238849 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.238723 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj749\" (UniqueName: \"kubernetes.io/projected/8dc0227f-b22b-4207-828a-415efd0d2b9d-kube-api-access-qj749\") pod \"llmisvc-controller-manager-777c6bd7d5-xcvtc\" (UID: \"8dc0227f-b22b-4207-828a-415efd0d2b9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:51:48.339920 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.339883 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc0227f-b22b-4207-828a-415efd0d2b9d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xcvtc\" (UID: \"8dc0227f-b22b-4207-828a-415efd0d2b9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:51:48.340084 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.339955 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86rg7\" (UniqueName: \"kubernetes.io/projected/67976ed9-c2b3-4983-8196-f22ec66af6bd-kube-api-access-86rg7\") pod \"kserve-controller-manager-d6f4b65b8-lsdvj\" (UID: \"67976ed9-c2b3-4983-8196-f22ec66af6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:48.340084 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.339982 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj749\" (UniqueName: \"kubernetes.io/projected/8dc0227f-b22b-4207-828a-415efd0d2b9d-kube-api-access-qj749\") pod \"llmisvc-controller-manager-777c6bd7d5-xcvtc\" (UID: \"8dc0227f-b22b-4207-828a-415efd0d2b9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:51:48.340084 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.340063 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67976ed9-c2b3-4983-8196-f22ec66af6bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-lsdvj\" (UID: \"67976ed9-c2b3-4983-8196-f22ec66af6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:48.340258 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:51:48.340181 2571 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 10 12:51:48.340258 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:51:48.340241 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67976ed9-c2b3-4983-8196-f22ec66af6bd-cert podName:67976ed9-c2b3-4983-8196-f22ec66af6bd nodeName:}" failed. No retries permitted until 2026-07-10 12:51:48.840222925 +0000 UTC m=+359.725983520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/67976ed9-c2b3-4983-8196-f22ec66af6bd-cert") pod "kserve-controller-manager-d6f4b65b8-lsdvj" (UID: "67976ed9-c2b3-4983-8196-f22ec66af6bd") : secret "kserve-webhook-server-cert" not found Jul 10 12:51:48.342606 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.342558 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dc0227f-b22b-4207-828a-415efd0d2b9d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xcvtc\" (UID: \"8dc0227f-b22b-4207-828a-415efd0d2b9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:51:48.352563 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.352498 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj749\" (UniqueName: \"kubernetes.io/projected/8dc0227f-b22b-4207-828a-415efd0d2b9d-kube-api-access-qj749\") pod \"llmisvc-controller-manager-777c6bd7d5-xcvtc\" (UID: \"8dc0227f-b22b-4207-828a-415efd0d2b9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:51:48.352645 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.352631 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86rg7\" (UniqueName: \"kubernetes.io/projected/67976ed9-c2b3-4983-8196-f22ec66af6bd-kube-api-access-86rg7\") pod \"kserve-controller-manager-d6f4b65b8-lsdvj\" (UID: \"67976ed9-c2b3-4983-8196-f22ec66af6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:48.446036 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.445983 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:51:48.846490 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.846444 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67976ed9-c2b3-4983-8196-f22ec66af6bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-lsdvj\" (UID: \"67976ed9-c2b3-4983-8196-f22ec66af6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:48.849675 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.849635 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67976ed9-c2b3-4983-8196-f22ec66af6bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-lsdvj\" (UID: \"67976ed9-c2b3-4983-8196-f22ec66af6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:48.889749 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.889716 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:51:48.889866 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.889813 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:51:48.889948 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.889873 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:51:48.964592 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:48.964565 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc"] Jul 10 12:51:48.966064 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:48.966042 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8dc0227f_b22b_4207_828a_415efd0d2b9d.slice/crio-a82f8f6cf399f42b9a18fedb8253d05a6c959c5959e29f881b09b08a64bf4a66 WatchSource:0}: Error finding container a82f8f6cf399f42b9a18fedb8253d05a6c959c5959e29f881b09b08a64bf4a66: Status 404 returned error can't find the container with id a82f8f6cf399f42b9a18fedb8253d05a6c959c5959e29f881b09b08a64bf4a66 Jul 10 12:51:49.022416 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:49.022390 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:49.181229 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:49.181187 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" event={"ID":"fbd98ae2-6a89-4ad5-8614-96ea5f071f72","Type":"ContainerStarted","Data":"b097263846149a2f57037adaf44c81a7095c6c375d31154d595e1e6c800f3e96"} Jul 10 12:51:49.182543 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:49.182519 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" event={"ID":"8dc0227f-b22b-4207-828a-415efd0d2b9d","Type":"ContainerStarted","Data":"a82f8f6cf399f42b9a18fedb8253d05a6c959c5959e29f881b09b08a64bf4a66"} Jul 10 12:51:49.190740 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:49.190719 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj"] Jul 10 12:51:49.193661 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:51:49.193635 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67976ed9_c2b3_4983_8196_f22ec66af6bd.slice/crio-4811ae4d5edce25d9650c2ecd66795f61aece340746a4687e0094e6e6abe33be WatchSource:0}: Error finding container 4811ae4d5edce25d9650c2ecd66795f61aece340746a4687e0094e6e6abe33be: Status 404 returned error can't find the container with id 4811ae4d5edce25d9650c2ecd66795f61aece340746a4687e0094e6e6abe33be Jul 10 12:51:49.252577 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:49.252532 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" podStartSLOduration=1.485611564 podStartE2EDuration="4.252517298s" totalImagesPullingTime="2.766905734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:51:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:51:49.249333007 +0000 UTC m=+360.135093623" watchObservedRunningTime="2026-07-10 12:51:49.252517298 +0000 UTC m=+360.138277912" Jul 10 12:51:49.748157 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:49.748124 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7"] Jul 10 12:51:49.748415 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:49.748388 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" podUID="16602aec-ffe6-4507-a782-4c7edd04f6d1" containerName="discovery" containerID="cri-o://48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb" gracePeriod=30 Jul 10 12:51:49.965165 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:49.964947 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:49.972058 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:49.972020 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:50.131539 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.131267 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:50.195273 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.194536 2571 generic.go:356] "Generic (PLEG): container finished" podID="16602aec-ffe6-4507-a782-4c7edd04f6d1" containerID="48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb" exitCode=0 Jul 10 12:51:50.195273 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.194630 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" event={"ID":"16602aec-ffe6-4507-a782-4c7edd04f6d1","Type":"ContainerDied","Data":"48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb"} Jul 10 12:51:50.195273 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.194680 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" event={"ID":"16602aec-ffe6-4507-a782-4c7edd04f6d1","Type":"ContainerDied","Data":"35ce50058dba519f690495d0fa8af3e3e45b7d047d8f8bb50e66c0308148a203"} Jul 10 12:51:50.195273 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.194693 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7" Jul 10 12:51:50.195273 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.194709 2571 scope.go:117] "RemoveContainer" containerID="48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb" Jul 10 12:51:50.199044 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.198778 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" event={"ID":"67976ed9-c2b3-4983-8196-f22ec66af6bd","Type":"ContainerStarted","Data":"4811ae4d5edce25d9650c2ecd66795f61aece340746a4687e0094e6e6abe33be"} Jul 10 12:51:50.199044 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.198837 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:50.199921 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.199899 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-695cd9447dsp57g" Jul 10 12:51:50.264633 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.264599 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-ca-configmap\") pod \"16602aec-ffe6-4507-a782-4c7edd04f6d1\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " Jul 10 12:51:50.264809 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.264657 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-dns-cert\") pod \"16602aec-ffe6-4507-a782-4c7edd04f6d1\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " Jul 10 12:51:50.264809 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.264726 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhqzx\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-kube-api-access-lhqzx\") pod \"16602aec-ffe6-4507-a782-4c7edd04f6d1\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " Jul 10 12:51:50.264809 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.264767 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/16602aec-ffe6-4507-a782-4c7edd04f6d1-local-certs\") pod \"16602aec-ffe6-4507-a782-4c7edd04f6d1\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " Jul 10 12:51:50.264809 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.264795 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-kubeconfig\") pod \"16602aec-ffe6-4507-a782-4c7edd04f6d1\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " Jul 10 12:51:50.265026 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.264820 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-token\") pod \"16602aec-ffe6-4507-a782-4c7edd04f6d1\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " Jul 10 12:51:50.265026 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.264849 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-cacerts\") pod \"16602aec-ffe6-4507-a782-4c7edd04f6d1\" (UID: \"16602aec-ffe6-4507-a782-4c7edd04f6d1\") " Jul 10 12:51:50.265736 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.265362 2571 scope.go:117] "RemoveContainer" containerID="48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb" Jul 10 12:51:50.266803 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:51:50.266767 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb\": container with ID starting with 48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb not found: ID does not exist" containerID="48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb" Jul 10 12:51:50.266939 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.266776 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "16602aec-ffe6-4507-a782-4c7edd04f6d1" (UID: "16602aec-ffe6-4507-a782-4c7edd04f6d1"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:51:50.267411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.267367 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb"} err="failed to get container status \"48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb\": rpc error: code = NotFound desc = could not find container \"48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb\": container with ID starting with 48536973e1e653c92599b5307d1fd4f0f3988e2300ea9519b5977a29c599cadb not found: ID does not exist" Jul 10 12:51:50.270805 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.270722 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-kube-api-access-lhqzx" (OuterVolumeSpecName: "kube-api-access-lhqzx") pod "16602aec-ffe6-4507-a782-4c7edd04f6d1" (UID: "16602aec-ffe6-4507-a782-4c7edd04f6d1"). InnerVolumeSpecName "kube-api-access-lhqzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:51:50.270913 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.270791 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-cacerts" (OuterVolumeSpecName: "cacerts") pod "16602aec-ffe6-4507-a782-4c7edd04f6d1" (UID: "16602aec-ffe6-4507-a782-4c7edd04f6d1"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:51:50.270913 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.270852 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "16602aec-ffe6-4507-a782-4c7edd04f6d1" (UID: "16602aec-ffe6-4507-a782-4c7edd04f6d1"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:51:50.271203 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.271175 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16602aec-ffe6-4507-a782-4c7edd04f6d1-local-certs" (OuterVolumeSpecName: "local-certs") pod "16602aec-ffe6-4507-a782-4c7edd04f6d1" (UID: "16602aec-ffe6-4507-a782-4c7edd04f6d1"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:51:50.271845 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.271802 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "16602aec-ffe6-4507-a782-4c7edd04f6d1" (UID: "16602aec-ffe6-4507-a782-4c7edd04f6d1"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:51:50.272622 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.272206 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "16602aec-ffe6-4507-a782-4c7edd04f6d1" (UID: "16602aec-ffe6-4507-a782-4c7edd04f6d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:51:50.366672 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.366547 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhqzx\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-kube-api-access-lhqzx\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:50.366672 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.366606 2571 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/16602aec-ffe6-4507-a782-4c7edd04f6d1-local-certs\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:50.366672 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.366625 2571 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-kubeconfig\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:50.366672 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.366638 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-token\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:50.366672 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.366651 2571 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-cacerts\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:50.366672 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.366663 2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-ca-configmap\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:50.366672 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.366676 2571 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/16602aec-ffe6-4507-a782-4c7edd04f6d1-istio-csr-dns-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:51:50.522566 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.522533 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7"] Jul 10 12:51:50.528888 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:50.528862 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zdrf7"] Jul 10 12:51:51.536571 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:51.536535 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16602aec-ffe6-4507-a782-4c7edd04f6d1" path="/var/lib/kubelet/pods/16602aec-ffe6-4507-a782-4c7edd04f6d1/volumes" Jul 10 12:51:53.217705 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:53.217666 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" event={"ID":"8dc0227f-b22b-4207-828a-415efd0d2b9d","Type":"ContainerStarted","Data":"b5a8600e9b34ea0ac13c8de9fa816c22e54774265a9814c03eae8678b5004141"} Jul 10 12:51:53.218234 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:53.217762 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:51:53.219424 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:53.219398 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" event={"ID":"67976ed9-c2b3-4983-8196-f22ec66af6bd","Type":"ContainerStarted","Data":"263a2d97c8105dc2738224627d9224fecd143b95706dff1207aa270f157232ca"} Jul 10 12:51:53.219555 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:53.219459 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:51:53.237030 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:53.236990 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" podStartSLOduration=1.5744544729999999 podStartE2EDuration="5.236977905s" totalImagesPullingTime="3.662523432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:51:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:51:53.235552042 +0000 UTC m=+364.121312656" watchObservedRunningTime="2026-07-10 12:51:53.236977905 +0000 UTC m=+364.122738511" Jul 10 12:51:53.253232 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:53.253189 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" podStartSLOduration=1.814211336 podStartE2EDuration="5.253176969s" totalImagesPullingTime="3.438965633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:51:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:51:53.251921172 +0000 UTC m=+364.137681802" watchObservedRunningTime="2026-07-10 12:51:53.253176969 +0000 UTC m=+364.138937586" Jul 10 12:51:55.156161 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:55.156134 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-5q4nh" Jul 10 12:51:55.157890 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:51:55.157867 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dfc9h" Jul 10 12:52:24.227353 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:24.227326 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xcvtc" Jul 10 12:52:24.230254 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:24.230238 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lsdvj" Jul 10 12:52:38.378856 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.378817 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jtk4x"] Jul 10 12:52:38.379493 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.379412 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16602aec-ffe6-4507-a782-4c7edd04f6d1" containerName="discovery" Jul 10 12:52:38.379493 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.379433 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="16602aec-ffe6-4507-a782-4c7edd04f6d1" containerName="discovery" Jul 10 12:52:38.379629 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.379537 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="16602aec-ffe6-4507-a782-4c7edd04f6d1" containerName="discovery" Jul 10 12:52:38.397006 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.396980 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jtk4x"] Jul 10 12:52:38.397153 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.397094 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" Jul 10 12:52:38.399696 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.399672 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 12:52:38.400695 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.400669 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-5htkr\"" Jul 10 12:52:38.400969 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.400951 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 12:52:38.466079 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.466055 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv5bg\" (UniqueName: \"kubernetes.io/projected/c778be57-f8c1-4b67-a48b-5b56499d5e4d-kube-api-access-lv5bg\") pod \"kuadrant-operator-catalog-jtk4x\" (UID: \"c778be57-f8c1-4b67-a48b-5b56499d5e4d\") " pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" Jul 10 12:52:38.566724 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.566694 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv5bg\" (UniqueName: \"kubernetes.io/projected/c778be57-f8c1-4b67-a48b-5b56499d5e4d-kube-api-access-lv5bg\") pod \"kuadrant-operator-catalog-jtk4x\" (UID: \"c778be57-f8c1-4b67-a48b-5b56499d5e4d\") " pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" Jul 10 12:52:38.575393 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.575371 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv5bg\" (UniqueName: \"kubernetes.io/projected/c778be57-f8c1-4b67-a48b-5b56499d5e4d-kube-api-access-lv5bg\") pod \"kuadrant-operator-catalog-jtk4x\" (UID: \"c778be57-f8c1-4b67-a48b-5b56499d5e4d\") " pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" Jul 10 12:52:38.709709 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.709684 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" Jul 10 12:52:38.746153 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.746130 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jtk4x"] Jul 10 12:52:38.838508 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.838483 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jtk4x"] Jul 10 12:52:38.840885 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:52:38.840859 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc778be57_f8c1_4b67_a48b_5b56499d5e4d.slice/crio-807a2d7c1cb245999593c587bb3f99e0c77620be13676e58af0a87d639f2076c WatchSource:0}: Error finding container 807a2d7c1cb245999593c587bb3f99e0c77620be13676e58af0a87d639f2076c: Status 404 returned error can't find the container with id 807a2d7c1cb245999593c587bb3f99e0c77620be13676e58af0a87d639f2076c Jul 10 12:52:38.957417 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.957386 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j9wqv"] Jul 10 12:52:38.962178 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.962134 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" Jul 10 12:52:38.968234 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:38.968212 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j9wqv"] Jul 10 12:52:39.071113 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:39.071077 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsh96\" (UniqueName: \"kubernetes.io/projected/f32cef55-66bb-4a77-be3a-aaeb6084bcc8-kube-api-access-rsh96\") pod \"kuadrant-operator-catalog-j9wqv\" (UID: \"f32cef55-66bb-4a77-be3a-aaeb6084bcc8\") " pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" Jul 10 12:52:39.172057 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:39.172028 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsh96\" (UniqueName: \"kubernetes.io/projected/f32cef55-66bb-4a77-be3a-aaeb6084bcc8-kube-api-access-rsh96\") pod \"kuadrant-operator-catalog-j9wqv\" (UID: \"f32cef55-66bb-4a77-be3a-aaeb6084bcc8\") " pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" Jul 10 12:52:39.180033 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:39.180011 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsh96\" (UniqueName: \"kubernetes.io/projected/f32cef55-66bb-4a77-be3a-aaeb6084bcc8-kube-api-access-rsh96\") pod \"kuadrant-operator-catalog-j9wqv\" (UID: \"f32cef55-66bb-4a77-be3a-aaeb6084bcc8\") " pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" Jul 10 12:52:39.273325 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:39.273225 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" Jul 10 12:52:39.417864 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:39.417831 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j9wqv"] Jul 10 12:52:39.449159 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:52:39.449130 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32cef55_66bb_4a77_be3a_aaeb6084bcc8.slice/crio-189de7d401e7fdbfccf7ab0b32b57f0be869b5a5c12106a3437012cbae645c79 WatchSource:0}: Error finding container 189de7d401e7fdbfccf7ab0b32b57f0be869b5a5c12106a3437012cbae645c79: Status 404 returned error can't find the container with id 189de7d401e7fdbfccf7ab0b32b57f0be869b5a5c12106a3437012cbae645c79 Jul 10 12:52:39.453827 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:39.453791 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" event={"ID":"c778be57-f8c1-4b67-a48b-5b56499d5e4d","Type":"ContainerStarted","Data":"807a2d7c1cb245999593c587bb3f99e0c77620be13676e58af0a87d639f2076c"} Jul 10 12:52:40.462064 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:40.462027 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" event={"ID":"f32cef55-66bb-4a77-be3a-aaeb6084bcc8","Type":"ContainerStarted","Data":"189de7d401e7fdbfccf7ab0b32b57f0be869b5a5c12106a3437012cbae645c79"} Jul 10 12:52:41.468960 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:41.468917 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" event={"ID":"c778be57-f8c1-4b67-a48b-5b56499d5e4d","Type":"ContainerStarted","Data":"cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c"} Jul 10 12:52:41.469435 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:41.468953 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" podUID="c778be57-f8c1-4b67-a48b-5b56499d5e4d" containerName="registry-server" containerID="cri-o://cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c" gracePeriod=2 Jul 10 12:52:41.470693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:41.470663 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" event={"ID":"f32cef55-66bb-4a77-be3a-aaeb6084bcc8","Type":"ContainerStarted","Data":"0334258b23da083fd8168cfd6e666ae10704d9e698f5452d1c899e8e6099e3f6"} Jul 10 12:52:41.488937 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:41.488893 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" podStartSLOduration=1.341051811 podStartE2EDuration="3.488881162s" totalImagesPullingTime="2.147829351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:52:41.48721533 +0000 UTC m=+412.372975944" watchObservedRunningTime="2026-07-10 12:52:41.488881162 +0000 UTC m=+412.374641778" Jul 10 12:52:41.504308 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:41.504252 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" podStartSLOduration=1.961571362 podStartE2EDuration="3.504237432s" totalImagesPullingTime="1.54266607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:52:41.502927444 +0000 UTC m=+412.388688059" watchObservedRunningTime="2026-07-10 12:52:41.504237432 +0000 UTC m=+412.389998046" Jul 10 12:52:41.719555 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:41.719496 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" Jul 10 12:52:41.801354 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:41.801321 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lv5bg\" (UniqueName: \"kubernetes.io/projected/c778be57-f8c1-4b67-a48b-5b56499d5e4d-kube-api-access-lv5bg\") pod \"c778be57-f8c1-4b67-a48b-5b56499d5e4d\" (UID: \"c778be57-f8c1-4b67-a48b-5b56499d5e4d\") " Jul 10 12:52:41.803470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:41.803444 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c778be57-f8c1-4b67-a48b-5b56499d5e4d-kube-api-access-lv5bg" (OuterVolumeSpecName: "kube-api-access-lv5bg") pod "c778be57-f8c1-4b67-a48b-5b56499d5e4d" (UID: "c778be57-f8c1-4b67-a48b-5b56499d5e4d"). InnerVolumeSpecName "kube-api-access-lv5bg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:52:41.902990 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:41.902960 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv5bg\" (UniqueName: \"kubernetes.io/projected/c778be57-f8c1-4b67-a48b-5b56499d5e4d-kube-api-access-lv5bg\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:42.477154 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:42.477118 2571 generic.go:356] "Generic (PLEG): container finished" podID="c778be57-f8c1-4b67-a48b-5b56499d5e4d" containerID="cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c" exitCode=0 Jul 10 12:52:42.477642 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:42.477159 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" Jul 10 12:52:42.477642 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:42.477201 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" event={"ID":"c778be57-f8c1-4b67-a48b-5b56499d5e4d","Type":"ContainerDied","Data":"cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c"} Jul 10 12:52:42.477642 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:42.477254 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jtk4x" event={"ID":"c778be57-f8c1-4b67-a48b-5b56499d5e4d","Type":"ContainerDied","Data":"807a2d7c1cb245999593c587bb3f99e0c77620be13676e58af0a87d639f2076c"} Jul 10 12:52:42.477642 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:42.477277 2571 scope.go:117] "RemoveContainer" containerID="cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c" Jul 10 12:52:42.486548 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:42.486537 2571 scope.go:117] "RemoveContainer" containerID="cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c" Jul 10 12:52:42.486793 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:52:42.486777 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c\": container with ID starting with cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c not found: ID does not exist" containerID="cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c" Jul 10 12:52:42.486831 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:42.486801 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c"} err="failed to get container status \"cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c\": rpc error: code = NotFound desc = could not find container \"cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c\": container with ID starting with cd3a476c4cf3453a039d6be636b8af6e17039ae0539b2dea1f35e2ae44fc017c not found: ID does not exist" Jul 10 12:52:42.498793 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:42.498775 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jtk4x"] Jul 10 12:52:42.502855 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:42.502834 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jtk4x"] Jul 10 12:52:43.535389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:43.535360 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c778be57-f8c1-4b67-a48b-5b56499d5e4d" path="/var/lib/kubelet/pods/c778be57-f8c1-4b67-a48b-5b56499d5e4d/volumes" Jul 10 12:52:49.273793 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:49.273753 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" Jul 10 12:52:49.274240 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:49.273805 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" Jul 10 12:52:49.296032 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:49.296004 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" Jul 10 12:52:49.537861 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:49.537786 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-j9wqv" Jul 10 12:52:53.791394 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.791359 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk"] Jul 10 12:52:53.791765 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.791748 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c778be57-f8c1-4b67-a48b-5b56499d5e4d" containerName="registry-server" Jul 10 12:52:53.791765 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.791759 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c778be57-f8c1-4b67-a48b-5b56499d5e4d" containerName="registry-server" Jul 10 12:52:53.791838 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.791822 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c778be57-f8c1-4b67-a48b-5b56499d5e4d" containerName="registry-server" Jul 10 12:52:53.796753 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.796736 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:53.799395 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.799371 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6qgc9\"" Jul 10 12:52:53.803872 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.803847 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk"] Jul 10 12:52:53.908972 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.908935 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:53.908972 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.908978 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:53.909187 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:53.909062 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4hnr\" (UniqueName: \"kubernetes.io/projected/794c8a39-094d-45a2-b5fa-48da12241ccc-kube-api-access-r4hnr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:54.009550 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.009517 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4hnr\" (UniqueName: \"kubernetes.io/projected/794c8a39-094d-45a2-b5fa-48da12241ccc-kube-api-access-r4hnr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:54.009744 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.009683 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:54.009744 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.009728 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:54.010028 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.010011 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:54.010077 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.010061 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:54.018460 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.018432 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4hnr\" (UniqueName: \"kubernetes.io/projected/794c8a39-094d-45a2-b5fa-48da12241ccc-kube-api-access-r4hnr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:54.108404 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.108329 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:54.241827 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.241806 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk"] Jul 10 12:52:54.243586 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:52:54.243557 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794c8a39_094d_45a2_b5fa_48da12241ccc.slice/crio-d3441d7a5939fa81d319f9b9ebda8fb00c16e5b7a439f8a1297c553293025ab0 WatchSource:0}: Error finding container d3441d7a5939fa81d319f9b9ebda8fb00c16e5b7a439f8a1297c553293025ab0: Status 404 returned error can't find the container with id d3441d7a5939fa81d319f9b9ebda8fb00c16e5b7a439f8a1297c553293025ab0 Jul 10 12:52:54.392123 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.392097 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9"] Jul 10 12:52:54.395672 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.395657 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.404479 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.404458 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9"] Jul 10 12:52:54.514009 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.513979 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.514141 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.514029 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.514141 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.514046 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w746\" (UniqueName: \"kubernetes.io/projected/c077e5c6-39f9-4c23-a616-b6a1e3516aad-kube-api-access-8w746\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.536962 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.536932 2571 generic.go:356] "Generic (PLEG): container finished" podID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerID="27b7f3bc9745b62c6c1955aabe470bf5864b53c803018ce1cfa2bfb0705593a4" exitCode=0 Jul 10 12:52:54.537054 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.537019 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" event={"ID":"794c8a39-094d-45a2-b5fa-48da12241ccc","Type":"ContainerDied","Data":"27b7f3bc9745b62c6c1955aabe470bf5864b53c803018ce1cfa2bfb0705593a4"} Jul 10 12:52:54.537099 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.537062 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" event={"ID":"794c8a39-094d-45a2-b5fa-48da12241ccc","Type":"ContainerStarted","Data":"d3441d7a5939fa81d319f9b9ebda8fb00c16e5b7a439f8a1297c553293025ab0"} Jul 10 12:52:54.614716 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.614688 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.614876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.614747 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.614876 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.614772 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w746\" (UniqueName: \"kubernetes.io/projected/c077e5c6-39f9-4c23-a616-b6a1e3516aad-kube-api-access-8w746\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.615160 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.615133 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.615160 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.615150 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.623792 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.623766 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w746\" (UniqueName: \"kubernetes.io/projected/c077e5c6-39f9-4c23-a616-b6a1e3516aad-kube-api-access-8w746\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.723622 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.723530 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:54.788600 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.788570 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n"] Jul 10 12:52:54.793560 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.793541 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:54.806449 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.806420 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n"] Jul 10 12:52:54.861093 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.861058 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9"] Jul 10 12:52:54.863902 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:52:54.863872 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc077e5c6_39f9_4c23_a616_b6a1e3516aad.slice/crio-7e7aa8b52ebcc90a394b21963d146aec699cef4d7dfbb51ca466bced5e39f94b WatchSource:0}: Error finding container 7e7aa8b52ebcc90a394b21963d146aec699cef4d7dfbb51ca466bced5e39f94b: Status 404 returned error can't find the container with id 7e7aa8b52ebcc90a394b21963d146aec699cef4d7dfbb51ca466bced5e39f94b Jul 10 12:52:54.917658 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.917632 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:54.917764 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.917681 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9mxc\" (UniqueName: \"kubernetes.io/projected/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-kube-api-access-m9mxc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:54.917764 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:54.917704 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:55.018915 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.018845 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:55.018915 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.018897 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9mxc\" (UniqueName: \"kubernetes.io/projected/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-kube-api-access-m9mxc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:55.018915 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.018917 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:55.019234 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.019215 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:55.019276 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.019260 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:55.027753 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.027730 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9mxc\" (UniqueName: \"kubernetes.io/projected/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-kube-api-access-m9mxc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:55.109417 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.109392 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:55.197850 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.197128 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf"] Jul 10 12:52:55.202510 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.202478 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.209970 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.209887 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf"] Jul 10 12:52:55.256362 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.256340 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n"] Jul 10 12:52:55.281014 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:52:55.280957 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad99d4fa_8e2a_4804_b7e6_d33c458d8f8f.slice/crio-f9729d22c9336002540734fb520632203a8bf39397e5a3a1345f1fca4d6786b2 WatchSource:0}: Error finding container f9729d22c9336002540734fb520632203a8bf39397e5a3a1345f1fca4d6786b2: Status 404 returned error can't find the container with id f9729d22c9336002540734fb520632203a8bf39397e5a3a1345f1fca4d6786b2 Jul 10 12:52:55.321327 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.321302 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qphph\" (UniqueName: \"kubernetes.io/projected/9ea73571-bdb4-443f-9188-2608105f000a-kube-api-access-qphph\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.321441 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.321361 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.321441 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.321393 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.421828 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.421801 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.421943 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.421844 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.421943 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.421920 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qphph\" (UniqueName: \"kubernetes.io/projected/9ea73571-bdb4-443f-9188-2608105f000a-kube-api-access-qphph\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.422199 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.422178 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.422262 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.422195 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.430208 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.430185 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qphph\" (UniqueName: \"kubernetes.io/projected/9ea73571-bdb4-443f-9188-2608105f000a-kube-api-access-qphph\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.516908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.516883 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:55.545090 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.545026 2571 generic.go:356] "Generic (PLEG): container finished" podID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerID="07f806aeb15187868cba577d338ad93110236d3ee0fc6248bf9a654f22223382" exitCode=0 Jul 10 12:52:55.545205 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.545096 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" event={"ID":"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f","Type":"ContainerDied","Data":"07f806aeb15187868cba577d338ad93110236d3ee0fc6248bf9a654f22223382"} Jul 10 12:52:55.545205 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.545146 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" event={"ID":"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f","Type":"ContainerStarted","Data":"f9729d22c9336002540734fb520632203a8bf39397e5a3a1345f1fca4d6786b2"} Jul 10 12:52:55.546844 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.546817 2571 generic.go:356] "Generic (PLEG): container finished" podID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerID="9062e8bf9989b4bbc160bc02e2787e2e339cba65d8e3d0d0a6cf132b8c120162" exitCode=0 Jul 10 12:52:55.546997 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.546880 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" event={"ID":"c077e5c6-39f9-4c23-a616-b6a1e3516aad","Type":"ContainerDied","Data":"9062e8bf9989b4bbc160bc02e2787e2e339cba65d8e3d0d0a6cf132b8c120162"} Jul 10 12:52:55.546997 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.546922 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" event={"ID":"c077e5c6-39f9-4c23-a616-b6a1e3516aad","Type":"ContainerStarted","Data":"7e7aa8b52ebcc90a394b21963d146aec699cef4d7dfbb51ca466bced5e39f94b"} Jul 10 12:52:55.549094 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.549035 2571 generic.go:356] "Generic (PLEG): container finished" podID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerID="e6bc1481a039aaa44850165a3494bf50293dee9bc9afc455bf36e739294017d2" exitCode=0 Jul 10 12:52:55.549094 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.549087 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" event={"ID":"794c8a39-094d-45a2-b5fa-48da12241ccc","Type":"ContainerDied","Data":"e6bc1481a039aaa44850165a3494bf50293dee9bc9afc455bf36e739294017d2"} Jul 10 12:52:55.654989 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:55.654953 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf"] Jul 10 12:52:55.655978 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:52:55.655953 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea73571_bdb4_443f_9188_2608105f000a.slice/crio-5858b652020b1c849e2666fd688e1f70d4fffd72815e9d8bb0e0d2a3a26cc312 WatchSource:0}: Error finding container 5858b652020b1c849e2666fd688e1f70d4fffd72815e9d8bb0e0d2a3a26cc312: Status 404 returned error can't find the container with id 5858b652020b1c849e2666fd688e1f70d4fffd72815e9d8bb0e0d2a3a26cc312 Jul 10 12:52:56.556411 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:56.556381 2571 generic.go:356] "Generic (PLEG): container finished" podID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerID="45310f6792986fcd204bee633effccca758691c232ec376bec3b15c46c08c1f2" exitCode=0 Jul 10 12:52:56.556748 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:56.556462 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" event={"ID":"794c8a39-094d-45a2-b5fa-48da12241ccc","Type":"ContainerDied","Data":"45310f6792986fcd204bee633effccca758691c232ec376bec3b15c46c08c1f2"} Jul 10 12:52:56.558036 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:56.558015 2571 generic.go:356] "Generic (PLEG): container finished" podID="9ea73571-bdb4-443f-9188-2608105f000a" containerID="8d4ad51c1966d9b3da1f6cd030c20dc2ffe963544d867f8c3fe24909f19211ca" exitCode=0 Jul 10 12:52:56.558131 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:56.558107 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" event={"ID":"9ea73571-bdb4-443f-9188-2608105f000a","Type":"ContainerDied","Data":"8d4ad51c1966d9b3da1f6cd030c20dc2ffe963544d867f8c3fe24909f19211ca"} Jul 10 12:52:56.558187 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:56.558154 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" event={"ID":"9ea73571-bdb4-443f-9188-2608105f000a","Type":"ContainerStarted","Data":"5858b652020b1c849e2666fd688e1f70d4fffd72815e9d8bb0e0d2a3a26cc312"} Jul 10 12:52:56.560213 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:56.560185 2571 generic.go:356] "Generic (PLEG): container finished" podID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerID="397a145dc36d1d9fa8376c3d64d367b2171dc9b82e9a97fc69ba041ab4400d45" exitCode=0 Jul 10 12:52:56.560333 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:56.560216 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" event={"ID":"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f","Type":"ContainerDied","Data":"397a145dc36d1d9fa8376c3d64d367b2171dc9b82e9a97fc69ba041ab4400d45"} Jul 10 12:52:56.562232 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:56.562213 2571 generic.go:356] "Generic (PLEG): container finished" podID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerID="3e89bd03335534c1733df58ade0bfac2d60b116a4d6742b8f91a6b8395a1e804" exitCode=0 Jul 10 12:52:56.562328 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:56.562259 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" event={"ID":"c077e5c6-39f9-4c23-a616-b6a1e3516aad","Type":"ContainerDied","Data":"3e89bd03335534c1733df58ade0bfac2d60b116a4d6742b8f91a6b8395a1e804"} Jul 10 12:52:57.568954 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.568866 2571 generic.go:356] "Generic (PLEG): container finished" podID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerID="8f235753f7df7c76348b01fdcf027e25c78875f10b0d8e084a5366f543a70d91" exitCode=0 Jul 10 12:52:57.569388 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.568958 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" event={"ID":"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f","Type":"ContainerDied","Data":"8f235753f7df7c76348b01fdcf027e25c78875f10b0d8e084a5366f543a70d91"} Jul 10 12:52:57.570968 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.570947 2571 generic.go:356] "Generic (PLEG): container finished" podID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerID="3f85e4386c3ab8668c00c06b4dac6d2c222c8a44ab603fb3d68628f4bca2b399" exitCode=0 Jul 10 12:52:57.571089 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.571026 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" event={"ID":"c077e5c6-39f9-4c23-a616-b6a1e3516aad","Type":"ContainerDied","Data":"3f85e4386c3ab8668c00c06b4dac6d2c222c8a44ab603fb3d68628f4bca2b399"} Jul 10 12:52:57.572683 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.572662 2571 generic.go:356] "Generic (PLEG): container finished" podID="9ea73571-bdb4-443f-9188-2608105f000a" containerID="c949f32268cf75b0d54af394b16249f325e61fd0d3fe22bb517e449623b4a471" exitCode=0 Jul 10 12:52:57.572780 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.572738 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" event={"ID":"9ea73571-bdb4-443f-9188-2608105f000a","Type":"ContainerDied","Data":"c949f32268cf75b0d54af394b16249f325e61fd0d3fe22bb517e449623b4a471"} Jul 10 12:52:57.712987 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.712964 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:57.842707 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.842620 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4hnr\" (UniqueName: \"kubernetes.io/projected/794c8a39-094d-45a2-b5fa-48da12241ccc-kube-api-access-r4hnr\") pod \"794c8a39-094d-45a2-b5fa-48da12241ccc\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " Jul 10 12:52:57.842707 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.842706 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-util\") pod \"794c8a39-094d-45a2-b5fa-48da12241ccc\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " Jul 10 12:52:57.842902 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.842763 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-bundle\") pod \"794c8a39-094d-45a2-b5fa-48da12241ccc\" (UID: \"794c8a39-094d-45a2-b5fa-48da12241ccc\") " Jul 10 12:52:57.843281 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.843247 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-bundle" (OuterVolumeSpecName: "bundle") pod "794c8a39-094d-45a2-b5fa-48da12241ccc" (UID: "794c8a39-094d-45a2-b5fa-48da12241ccc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:52:57.844687 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.844660 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/794c8a39-094d-45a2-b5fa-48da12241ccc-kube-api-access-r4hnr" (OuterVolumeSpecName: "kube-api-access-r4hnr") pod "794c8a39-094d-45a2-b5fa-48da12241ccc" (UID: "794c8a39-094d-45a2-b5fa-48da12241ccc"). InnerVolumeSpecName "kube-api-access-r4hnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:52:57.848514 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.848491 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-util" (OuterVolumeSpecName: "util") pod "794c8a39-094d-45a2-b5fa-48da12241ccc" (UID: "794c8a39-094d-45a2-b5fa-48da12241ccc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:52:57.943584 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.943553 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-util\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:57.943584 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.943579 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/794c8a39-094d-45a2-b5fa-48da12241ccc-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:57.943584 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:57.943589 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4hnr\" (UniqueName: \"kubernetes.io/projected/794c8a39-094d-45a2-b5fa-48da12241ccc-kube-api-access-r4hnr\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:58.580220 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.580183 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" Jul 10 12:52:58.580220 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.580211 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk" event={"ID":"794c8a39-094d-45a2-b5fa-48da12241ccc","Type":"ContainerDied","Data":"d3441d7a5939fa81d319f9b9ebda8fb00c16e5b7a439f8a1297c553293025ab0"} Jul 10 12:52:58.580740 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.580265 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3441d7a5939fa81d319f9b9ebda8fb00c16e5b7a439f8a1297c553293025ab0" Jul 10 12:52:58.582382 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.582353 2571 generic.go:356] "Generic (PLEG): container finished" podID="9ea73571-bdb4-443f-9188-2608105f000a" containerID="960dcc957d02f14cdef53198e9517c95076a0760960736c5c2ae46eb54239e00" exitCode=0 Jul 10 12:52:58.582513 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.582385 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" event={"ID":"9ea73571-bdb4-443f-9188-2608105f000a","Type":"ContainerDied","Data":"960dcc957d02f14cdef53198e9517c95076a0760960736c5c2ae46eb54239e00"} Jul 10 12:52:58.739111 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.739091 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:58.741175 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.741159 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:58.851894 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.851819 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8w746\" (UniqueName: \"kubernetes.io/projected/c077e5c6-39f9-4c23-a616-b6a1e3516aad-kube-api-access-8w746\") pod \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " Jul 10 12:52:58.851894 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.851865 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-util\") pod \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " Jul 10 12:52:58.851894 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.851889 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-util\") pod \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " Jul 10 12:52:58.852160 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.851931 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-bundle\") pod \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " Jul 10 12:52:58.852160 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.851972 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-bundle\") pod \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\" (UID: \"c077e5c6-39f9-4c23-a616-b6a1e3516aad\") " Jul 10 12:52:58.852160 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.851999 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9mxc\" (UniqueName: \"kubernetes.io/projected/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-kube-api-access-m9mxc\") pod \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\" (UID: \"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f\") " Jul 10 12:52:58.852514 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.852487 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-bundle" (OuterVolumeSpecName: "bundle") pod "ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" (UID: "ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:52:58.852626 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.852487 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-bundle" (OuterVolumeSpecName: "bundle") pod "c077e5c6-39f9-4c23-a616-b6a1e3516aad" (UID: "c077e5c6-39f9-4c23-a616-b6a1e3516aad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:52:58.854514 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.854488 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c077e5c6-39f9-4c23-a616-b6a1e3516aad-kube-api-access-8w746" (OuterVolumeSpecName: "kube-api-access-8w746") pod "c077e5c6-39f9-4c23-a616-b6a1e3516aad" (UID: "c077e5c6-39f9-4c23-a616-b6a1e3516aad"). InnerVolumeSpecName "kube-api-access-8w746". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:52:58.854758 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.854727 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-kube-api-access-m9mxc" (OuterVolumeSpecName: "kube-api-access-m9mxc") pod "ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" (UID: "ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f"). InnerVolumeSpecName "kube-api-access-m9mxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:52:58.857922 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.857901 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-util" (OuterVolumeSpecName: "util") pod "c077e5c6-39f9-4c23-a616-b6a1e3516aad" (UID: "c077e5c6-39f9-4c23-a616-b6a1e3516aad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:52:58.858422 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.858403 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-util" (OuterVolumeSpecName: "util") pod "ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" (UID: "ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:52:58.953721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.953691 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w746\" (UniqueName: \"kubernetes.io/projected/c077e5c6-39f9-4c23-a616-b6a1e3516aad-kube-api-access-8w746\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:58.953721 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.953717 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-util\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:58.953857 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.953728 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-util\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:58.953857 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.953736 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:58.953857 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.953744 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c077e5c6-39f9-4c23-a616-b6a1e3516aad-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:58.953857 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:58.953751 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9mxc\" (UniqueName: \"kubernetes.io/projected/ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f-kube-api-access-m9mxc\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:59.590852 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.590822 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" Jul 10 12:52:59.591236 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.590844 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n" event={"ID":"ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f","Type":"ContainerDied","Data":"f9729d22c9336002540734fb520632203a8bf39397e5a3a1345f1fca4d6786b2"} Jul 10 12:52:59.591236 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.590898 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9729d22c9336002540734fb520632203a8bf39397e5a3a1345f1fca4d6786b2" Jul 10 12:52:59.592925 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.592902 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" Jul 10 12:52:59.592925 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.592911 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9" event={"ID":"c077e5c6-39f9-4c23-a616-b6a1e3516aad","Type":"ContainerDied","Data":"7e7aa8b52ebcc90a394b21963d146aec699cef4d7dfbb51ca466bced5e39f94b"} Jul 10 12:52:59.593100 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.592952 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e7aa8b52ebcc90a394b21963d146aec699cef4d7dfbb51ca466bced5e39f94b" Jul 10 12:52:59.737722 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.737695 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:52:59.862387 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.862275 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-util\") pod \"9ea73571-bdb4-443f-9188-2608105f000a\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " Jul 10 12:52:59.862387 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.862384 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-bundle\") pod \"9ea73571-bdb4-443f-9188-2608105f000a\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " Jul 10 12:52:59.862606 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.862424 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qphph\" (UniqueName: \"kubernetes.io/projected/9ea73571-bdb4-443f-9188-2608105f000a-kube-api-access-qphph\") pod \"9ea73571-bdb4-443f-9188-2608105f000a\" (UID: \"9ea73571-bdb4-443f-9188-2608105f000a\") " Jul 10 12:52:59.863069 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.863042 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-bundle" (OuterVolumeSpecName: "bundle") pod "9ea73571-bdb4-443f-9188-2608105f000a" (UID: "9ea73571-bdb4-443f-9188-2608105f000a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:52:59.864589 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.864567 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ea73571-bdb4-443f-9188-2608105f000a-kube-api-access-qphph" (OuterVolumeSpecName: "kube-api-access-qphph") pod "9ea73571-bdb4-443f-9188-2608105f000a" (UID: "9ea73571-bdb4-443f-9188-2608105f000a"). InnerVolumeSpecName "kube-api-access-qphph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:52:59.868543 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.868508 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-util" (OuterVolumeSpecName: "util") pod "9ea73571-bdb4-443f-9188-2608105f000a" (UID: "9ea73571-bdb4-443f-9188-2608105f000a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:52:59.963751 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.963718 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-util\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:59.963751 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.963743 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9ea73571-bdb4-443f-9188-2608105f000a-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:52:59.963751 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:52:59.963752 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qphph\" (UniqueName: \"kubernetes.io/projected/9ea73571-bdb4-443f-9188-2608105f000a-kube-api-access-qphph\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:53:00.606228 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:00.606199 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" Jul 10 12:53:00.606642 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:00.606237 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf" event={"ID":"9ea73571-bdb4-443f-9188-2608105f000a","Type":"ContainerDied","Data":"5858b652020b1c849e2666fd688e1f70d4fffd72815e9d8bb0e0d2a3a26cc312"} Jul 10 12:53:00.606642 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:00.606334 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5858b652020b1c849e2666fd688e1f70d4fffd72815e9d8bb0e0d2a3a26cc312" Jul 10 12:53:10.601460 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601432 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-frszx"] Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601874 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ea73571-bdb4-443f-9188-2608105f000a" containerName="extract" Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601887 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea73571-bdb4-443f-9188-2608105f000a" containerName="extract" Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601897 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerName="util" Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601902 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerName="util" Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601911 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerName="pull" Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601917 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerName="pull" Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601925 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerName="util" Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601930 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerName="util" Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601942 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ea73571-bdb4-443f-9188-2608105f000a" containerName="util" Jul 10 12:53:10.601944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601948 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea73571-bdb4-443f-9188-2608105f000a" containerName="util" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601959 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerName="util" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601964 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerName="util" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601975 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerName="pull" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601981 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerName="pull" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601989 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerName="extract" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.601994 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerName="extract" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602001 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerName="pull" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602006 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerName="pull" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602015 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerName="extract" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602019 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerName="extract" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602028 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ea73571-bdb4-443f-9188-2608105f000a" containerName="pull" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602033 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea73571-bdb4-443f-9188-2608105f000a" containerName="pull" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602040 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerName="extract" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602045 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerName="extract" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602098 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c077e5c6-39f9-4c23-a616-b6a1e3516aad" containerName="extract" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602106 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f" containerName="extract" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602116 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="794c8a39-094d-45a2-b5fa-48da12241ccc" containerName="extract" Jul 10 12:53:10.602285 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.602122 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ea73571-bdb4-443f-9188-2608105f000a" containerName="extract" Jul 10 12:53:10.606280 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.606265 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" Jul 10 12:53:10.609779 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.609761 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-s7p8m\"" Jul 10 12:53:10.617186 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.617166 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-frszx"] Jul 10 12:53:10.754624 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.754590 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkpsd\" (UniqueName: \"kubernetes.io/projected/4a1159c1-17bb-4050-99cf-bf53f01744bc-kube-api-access-xkpsd\") pod \"authorino-operator-75cc8d5cb7-frszx\" (UID: \"4a1159c1-17bb-4050-99cf-bf53f01744bc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" Jul 10 12:53:10.855455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.855382 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkpsd\" (UniqueName: \"kubernetes.io/projected/4a1159c1-17bb-4050-99cf-bf53f01744bc-kube-api-access-xkpsd\") pod \"authorino-operator-75cc8d5cb7-frszx\" (UID: \"4a1159c1-17bb-4050-99cf-bf53f01744bc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" Jul 10 12:53:10.867208 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.867184 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkpsd\" (UniqueName: \"kubernetes.io/projected/4a1159c1-17bb-4050-99cf-bf53f01744bc-kube-api-access-xkpsd\") pod \"authorino-operator-75cc8d5cb7-frszx\" (UID: \"4a1159c1-17bb-4050-99cf-bf53f01744bc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" Jul 10 12:53:10.917334 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:10.917286 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" Jul 10 12:53:11.048970 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:11.048947 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-frszx"] Jul 10 12:53:11.051052 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:53:11.051021 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a1159c1_17bb_4050_99cf_bf53f01744bc.slice/crio-2d81c0b77702838e91bde608fcac9149c2202579acfc072d2a1a41a26054a852 WatchSource:0}: Error finding container 2d81c0b77702838e91bde608fcac9149c2202579acfc072d2a1a41a26054a852: Status 404 returned error can't find the container with id 2d81c0b77702838e91bde608fcac9149c2202579acfc072d2a1a41a26054a852 Jul 10 12:53:11.667734 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:11.667694 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" event={"ID":"4a1159c1-17bb-4050-99cf-bf53f01744bc","Type":"ContainerStarted","Data":"2d81c0b77702838e91bde608fcac9149c2202579acfc072d2a1a41a26054a852"} Jul 10 12:53:13.681679 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:13.681647 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" event={"ID":"4a1159c1-17bb-4050-99cf-bf53f01744bc","Type":"ContainerStarted","Data":"ad1d543305162a2afeefc14afd8205dc78cd0411158d0e3893497dc546fc0ab3"} Jul 10 12:53:13.682063 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:13.681711 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" Jul 10 12:53:13.702793 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:13.702739 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" podStartSLOduration=1.461652362 podStartE2EDuration="3.70272214s" totalImagesPullingTime="2.241069778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:53:13.702646427 +0000 UTC m=+444.588407046" watchObservedRunningTime="2026-07-10 12:53:13.70272214 +0000 UTC m=+444.588482756" Jul 10 12:53:15.708974 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.708943 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr"] Jul 10 12:53:15.715421 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.715399 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:15.718531 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.718516 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8z5lv\"" Jul 10 12:53:15.725185 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.725160 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr"] Jul 10 12:53:15.811954 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.811909 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" (UID: \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:15.812074 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.811974 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c6wq\" (UniqueName: \"kubernetes.io/projected/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-kube-api-access-2c6wq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" (UID: \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:15.913244 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.913217 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" (UID: \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:15.913398 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.913254 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c6wq\" (UniqueName: \"kubernetes.io/projected/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-kube-api-access-2c6wq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" (UID: \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:15.913598 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.913581 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" (UID: \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:15.922189 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:15.922161 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c6wq\" (UniqueName: \"kubernetes.io/projected/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-kube-api-access-2c6wq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" (UID: \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:16.026584 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:16.026497 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:16.183489 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:16.183453 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr"] Jul 10 12:53:16.186724 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:53:16.186695 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c6eb05_1448_4c1c_a196_ae12f3f07fd1.slice/crio-fc0d549211bbea3a57d38beee357051b9e1fcfd339c7e2052e4148cd316c56c1 WatchSource:0}: Error finding container fc0d549211bbea3a57d38beee357051b9e1fcfd339c7e2052e4148cd316c56c1: Status 404 returned error can't find the container with id fc0d549211bbea3a57d38beee357051b9e1fcfd339c7e2052e4148cd316c56c1 Jul 10 12:53:16.698628 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:16.698588 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" event={"ID":"03c6eb05-1448-4c1c-a196-ae12f3f07fd1","Type":"ContainerStarted","Data":"fc0d549211bbea3a57d38beee357051b9e1fcfd339c7e2052e4148cd316c56c1"} Jul 10 12:53:18.516156 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:18.516123 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9"] Jul 10 12:53:18.519961 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:18.519938 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" Jul 10 12:53:18.523148 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:18.523125 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 10 12:53:18.523435 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:18.523416 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hvfzn\"" Jul 10 12:53:18.530376 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:18.530350 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9"] Jul 10 12:53:18.639530 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:18.639188 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md5q7\" (UniqueName: \"kubernetes.io/projected/fdc4f54b-a816-4e00-8624-ac3ba69fdc39-kube-api-access-md5q7\") pod \"dns-operator-controller-manager-864dd78c4c-pnqd9\" (UID: \"fdc4f54b-a816-4e00-8624-ac3ba69fdc39\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" Jul 10 12:53:18.740001 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:18.739972 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md5q7\" (UniqueName: \"kubernetes.io/projected/fdc4f54b-a816-4e00-8624-ac3ba69fdc39-kube-api-access-md5q7\") pod \"dns-operator-controller-manager-864dd78c4c-pnqd9\" (UID: \"fdc4f54b-a816-4e00-8624-ac3ba69fdc39\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" Jul 10 12:53:18.752640 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:18.752622 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md5q7\" (UniqueName: \"kubernetes.io/projected/fdc4f54b-a816-4e00-8624-ac3ba69fdc39-kube-api-access-md5q7\") pod \"dns-operator-controller-manager-864dd78c4c-pnqd9\" (UID: \"fdc4f54b-a816-4e00-8624-ac3ba69fdc39\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" Jul 10 12:53:18.831253 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:18.831187 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" Jul 10 12:53:19.005337 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:19.005282 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9"] Jul 10 12:53:19.008509 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:53:19.008472 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc4f54b_a816_4e00_8624_ac3ba69fdc39.slice/crio-c6ef722d2938ebd333abb0272cbb78d6a1a8dac35cbaa3c0dae7ff32b903651c WatchSource:0}: Error finding container c6ef722d2938ebd333abb0272cbb78d6a1a8dac35cbaa3c0dae7ff32b903651c: Status 404 returned error can't find the container with id c6ef722d2938ebd333abb0272cbb78d6a1a8dac35cbaa3c0dae7ff32b903651c Jul 10 12:53:19.720216 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:19.720187 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" event={"ID":"fdc4f54b-a816-4e00-8624-ac3ba69fdc39","Type":"ContainerStarted","Data":"c6ef722d2938ebd333abb0272cbb78d6a1a8dac35cbaa3c0dae7ff32b903651c"} Jul 10 12:53:20.925104 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:20.925060 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb"] Jul 10 12:53:20.928992 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:20.928975 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:20.931774 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:20.931750 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-96m6z\"" Jul 10 12:53:20.939909 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:20.939855 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb"] Jul 10 12:53:20.960742 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:20.960721 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t7cn\" (UniqueName: \"kubernetes.io/projected/7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f-kube-api-access-7t7cn\") pod \"limitador-operator-controller-manager-859d7b78bd-td6jb\" (UID: \"7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:21.062141 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:21.062101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t7cn\" (UniqueName: \"kubernetes.io/projected/7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f-kube-api-access-7t7cn\") pod \"limitador-operator-controller-manager-859d7b78bd-td6jb\" (UID: \"7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:21.082747 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:21.082720 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t7cn\" (UniqueName: \"kubernetes.io/projected/7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f-kube-api-access-7t7cn\") pod \"limitador-operator-controller-manager-859d7b78bd-td6jb\" (UID: \"7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:21.242495 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:21.242409 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:22.263621 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:22.263598 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb"] Jul 10 12:53:22.265621 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:53:22.265594 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7370e9fd_a2d2_47ff_8b82_4e4ec3333f7f.slice/crio-9bfa4c377b94926ef57a8bd0ac33af0f0732fdbdfc5122fd8be8eab09c62eb35 WatchSource:0}: Error finding container 9bfa4c377b94926ef57a8bd0ac33af0f0732fdbdfc5122fd8be8eab09c62eb35: Status 404 returned error can't find the container with id 9bfa4c377b94926ef57a8bd0ac33af0f0732fdbdfc5122fd8be8eab09c62eb35 Jul 10 12:53:22.743695 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:22.743654 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" event={"ID":"fdc4f54b-a816-4e00-8624-ac3ba69fdc39","Type":"ContainerStarted","Data":"d75180c104b45ff40ac4f9f0dccfb394a11b980cb95afb51c3388bc7de1946a0"} Jul 10 12:53:22.743873 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:22.743843 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" Jul 10 12:53:22.744899 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:22.744875 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" event={"ID":"7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f","Type":"ContainerStarted","Data":"9bfa4c377b94926ef57a8bd0ac33af0f0732fdbdfc5122fd8be8eab09c62eb35"} Jul 10 12:53:22.746385 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:22.746360 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" event={"ID":"03c6eb05-1448-4c1c-a196-ae12f3f07fd1","Type":"ContainerStarted","Data":"27bd5a523a7b3137b4d146bde1253b473c16e26d5b1c7f87d5576c333814867d"} Jul 10 12:53:22.746490 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:22.746415 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:22.762599 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:22.762546 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" podStartSLOduration=1.650205684 podStartE2EDuration="4.762533768s" totalImagesPullingTime="3.112328084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:53:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:53:22.759897024 +0000 UTC m=+453.645657638" watchObservedRunningTime="2026-07-10 12:53:22.762533768 +0000 UTC m=+453.648294383" Jul 10 12:53:22.776926 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:22.776886 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" podStartSLOduration=1.840777568 podStartE2EDuration="7.776876198s" totalImagesPullingTime="5.93609863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:53:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:53:22.776335027 +0000 UTC m=+453.662095642" watchObservedRunningTime="2026-07-10 12:53:22.776876198 +0000 UTC m=+453.662636814" Jul 10 12:53:24.689150 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:24.689125 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-frszx" Jul 10 12:53:24.760120 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:24.760079 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" event={"ID":"7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f","Type":"ContainerStarted","Data":"74a3e80ea41233cd4055d10a266920f4831a5fe5f5671b3c037dd53f730d7e13"} Jul 10 12:53:24.760316 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:24.760286 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:24.778244 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:24.778200 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" podStartSLOduration=3.051353221 podStartE2EDuration="4.778186001s" totalImagesPullingTime="1.72683278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:53:24.777485209 +0000 UTC m=+455.663245827" watchObservedRunningTime="2026-07-10 12:53:24.778186001 +0000 UTC m=+455.663946617" Jul 10 12:53:33.755052 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:33.755018 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:33.755586 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:33.755127 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-pnqd9" Jul 10 12:53:35.571978 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.571945 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr"] Jul 10 12:53:35.572391 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.572110 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" containerName="manager" containerID="cri-o://27bd5a523a7b3137b4d146bde1253b473c16e26d5b1c7f87d5576c333814867d" gracePeriod=2 Jul 10 12:53:35.578057 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.578030 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr"] Jul 10 12:53:35.593099 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.593075 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb"] Jul 10 12:53:35.593372 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.593323 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" containerName="manager" containerID="cri-o://74a3e80ea41233cd4055d10a266920f4831a5fe5f5671b3c037dd53f730d7e13" gracePeriod=2 Jul 10 12:53:35.596620 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.596485 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:35.596988 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.596930 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:35.599564 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.599450 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:35.599713 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.599693 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6"] Jul 10 12:53:35.600307 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.600264 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" containerName="manager" Jul 10 12:53:35.600307 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.600282 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" containerName="manager" Jul 10 12:53:35.600456 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.600419 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" containerName="manager" Jul 10 12:53:35.601906 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.601877 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:35.603829 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.603799 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb"] Jul 10 12:53:35.603920 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.603904 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:35.603967 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.603914 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:35.612960 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.612937 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6"] Jul 10 12:53:35.615364 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.615342 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb"] Jul 10 12:53:35.615773 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.615757 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" containerName="manager" Jul 10 12:53:35.615851 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.615777 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" containerName="manager" Jul 10 12:53:35.615944 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.615926 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" containerName="manager" Jul 10 12:53:35.619328 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.619311 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" Jul 10 12:53:35.631599 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.631574 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb"] Jul 10 12:53:35.635476 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.635432 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:35.637647 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.637622 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:35.639786 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.639756 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:35.641916 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.641894 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:35.685021 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.684997 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t9qn\" (UniqueName: \"kubernetes.io/projected/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-kube-api-access-5t9qn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8pkf6\" (UID: \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:35.685151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.685047 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg2gx\" (UniqueName: \"kubernetes.io/projected/3786890e-a8b9-4bbd-b1d9-e2354d8898b5-kube-api-access-bg2gx\") pod \"limitador-operator-controller-manager-859d7b78bd-fx7tb\" (UID: \"3786890e-a8b9-4bbd-b1d9-e2354d8898b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" Jul 10 12:53:35.685151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.685069 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8pkf6\" (UID: \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:35.785891 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.785856 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t9qn\" (UniqueName: \"kubernetes.io/projected/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-kube-api-access-5t9qn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8pkf6\" (UID: \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:35.786007 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.785925 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg2gx\" (UniqueName: \"kubernetes.io/projected/3786890e-a8b9-4bbd-b1d9-e2354d8898b5-kube-api-access-bg2gx\") pod \"limitador-operator-controller-manager-859d7b78bd-fx7tb\" (UID: \"3786890e-a8b9-4bbd-b1d9-e2354d8898b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" Jul 10 12:53:35.786088 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.786059 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8pkf6\" (UID: \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:35.786608 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.786584 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8pkf6\" (UID: \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:35.794590 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.794556 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg2gx\" (UniqueName: \"kubernetes.io/projected/3786890e-a8b9-4bbd-b1d9-e2354d8898b5-kube-api-access-bg2gx\") pod \"limitador-operator-controller-manager-859d7b78bd-fx7tb\" (UID: \"3786890e-a8b9-4bbd-b1d9-e2354d8898b5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" Jul 10 12:53:35.795146 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.795121 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t9qn\" (UniqueName: \"kubernetes.io/projected/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-kube-api-access-5t9qn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8pkf6\" (UID: \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:35.821166 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.821133 2571 generic.go:356] "Generic (PLEG): container finished" podID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" containerID="27bd5a523a7b3137b4d146bde1253b473c16e26d5b1c7f87d5576c333814867d" exitCode=0 Jul 10 12:53:35.823031 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.822958 2571 generic.go:356] "Generic (PLEG): container finished" podID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" containerID="74a3e80ea41233cd4055d10a266920f4831a5fe5f5671b3c037dd53f730d7e13" exitCode=0 Jul 10 12:53:35.841335 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.841317 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:35.843399 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.843383 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:35.843679 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.843661 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:35.845790 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.845765 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:35.847792 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.847771 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:35.849763 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:35.849741 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:35.887287 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.887262 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-extensions-socket-volume\") pod \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\" (UID: \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\") " Jul 10 12:53:35.887408 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.887347 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c6wq\" (UniqueName: \"kubernetes.io/projected/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-kube-api-access-2c6wq\") pod \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\" (UID: \"03c6eb05-1448-4c1c-a196-ae12f3f07fd1\") " Jul 10 12:53:35.887408 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.887394 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t7cn\" (UniqueName: \"kubernetes.io/projected/7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f-kube-api-access-7t7cn\") pod \"7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f\" (UID: \"7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f\") " Jul 10 12:53:35.887730 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.887705 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "03c6eb05-1448-4c1c-a196-ae12f3f07fd1" (UID: "03c6eb05-1448-4c1c-a196-ae12f3f07fd1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:53:35.889328 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.889305 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-kube-api-access-2c6wq" (OuterVolumeSpecName: "kube-api-access-2c6wq") pod "03c6eb05-1448-4c1c-a196-ae12f3f07fd1" (UID: "03c6eb05-1448-4c1c-a196-ae12f3f07fd1"). InnerVolumeSpecName "kube-api-access-2c6wq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:53:35.889394 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.889377 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f-kube-api-access-7t7cn" (OuterVolumeSpecName: "kube-api-access-7t7cn") pod "7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" (UID: "7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f"). InnerVolumeSpecName "kube-api-access-7t7cn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:53:35.988182 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.988149 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-extensions-socket-volume\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:53:35.988182 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.988175 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c6wq\" (UniqueName: \"kubernetes.io/projected/03c6eb05-1448-4c1c-a196-ae12f3f07fd1-kube-api-access-2c6wq\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:53:35.988182 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:35.988187 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t7cn\" (UniqueName: \"kubernetes.io/projected/7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f-kube-api-access-7t7cn\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:53:36.020523 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.020499 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:36.033109 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.033081 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" Jul 10 12:53:36.168462 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.168432 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6"] Jul 10 12:53:36.170510 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:53:36.170464 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7415d6a1_6e81_46f9_b257_8f6cffecc7a9.slice/crio-c278531da40d2c099afcaaafc84660addb6973432700e23802dfc3bdd54d28f2 WatchSource:0}: Error finding container c278531da40d2c099afcaaafc84660addb6973432700e23802dfc3bdd54d28f2: Status 404 returned error can't find the container with id c278531da40d2c099afcaaafc84660addb6973432700e23802dfc3bdd54d28f2 Jul 10 12:53:36.188349 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.188243 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb"] Jul 10 12:53:36.190155 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:53:36.190130 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3786890e_a8b9_4bbd_b1d9_e2354d8898b5.slice/crio-958adadc02081c0e61474d97136e8c8bb8bcb4eb44c9cba2168d21a4e8ba5038 WatchSource:0}: Error finding container 958adadc02081c0e61474d97136e8c8bb8bcb4eb44c9cba2168d21a4e8ba5038: Status 404 returned error can't find the container with id 958adadc02081c0e61474d97136e8c8bb8bcb4eb44c9cba2168d21a4e8ba5038 Jul 10 12:53:36.794037 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.794004 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g"] Jul 10 12:53:36.797644 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.797623 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:36.800233 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:36.800206 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:36.802424 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:36.802403 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:36.811850 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.811808 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g"] Jul 10 12:53:36.831631 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.831612 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:36.831771 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.831645 2571 scope.go:117] "RemoveContainer" containerID="74a3e80ea41233cd4055d10a266920f4831a5fe5f5671b3c037dd53f730d7e13" Jul 10 12:53:36.833815 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.833793 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" event={"ID":"7415d6a1-6e81-46f9-b257-8f6cffecc7a9","Type":"ContainerStarted","Data":"a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5"} Jul 10 12:53:36.834157 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.834063 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:36.834157 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.834104 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" event={"ID":"7415d6a1-6e81-46f9-b257-8f6cffecc7a9","Type":"ContainerStarted","Data":"c278531da40d2c099afcaaafc84660addb6973432700e23802dfc3bdd54d28f2"} Jul 10 12:53:36.834157 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:36.834049 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:36.835760 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.835734 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" event={"ID":"3786890e-a8b9-4bbd-b1d9-e2354d8898b5","Type":"ContainerStarted","Data":"5b02523010387c0aa4e1e2ef535eb640d9df0a0cacb698d731c225edb5d1ec2b"} Jul 10 12:53:36.835861 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.835786 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" Jul 10 12:53:36.835861 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.835810 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" event={"ID":"3786890e-a8b9-4bbd-b1d9-e2354d8898b5","Type":"ContainerStarted","Data":"958adadc02081c0e61474d97136e8c8bb8bcb4eb44c9cba2168d21a4e8ba5038"} Jul 10 12:53:36.836173 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:36.836131 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:36.837220 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.837205 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:36.838530 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:36.838206 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:36.845373 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.845354 2571 scope.go:117] "RemoveContainer" containerID="27bd5a523a7b3137b4d146bde1253b473c16e26d5b1c7f87d5576c333814867d" Jul 10 12:53:36.864127 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.864080 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" podStartSLOduration=1.864065922 podStartE2EDuration="1.864065922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:53:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:53:36.863480139 +0000 UTC m=+467.749240752" watchObservedRunningTime="2026-07-10 12:53:36.864065922 +0000 UTC m=+467.749826536" Jul 10 12:53:36.886509 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:36.886469 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:36.886947 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.886906 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" podStartSLOduration=1.886893917 podStartE2EDuration="1.886893917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:53:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:53:36.884512859 +0000 UTC m=+467.770273471" watchObservedRunningTime="2026-07-10 12:53:36.886893917 +0000 UTC m=+467.772654533" Jul 10 12:53:36.888317 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:36.888275 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-td6jb\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-td6jb" Jul 10 12:53:36.890206 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:36.890096 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-kjnsr\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-kjnsr" Jul 10 12:53:36.894587 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.894556 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e131bf8b-3e21-438c-9906-fe0c0c48d68a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nkw9g\" (UID: \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:36.894699 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.894602 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fxl5\" (UniqueName: \"kubernetes.io/projected/e131bf8b-3e21-438c-9906-fe0c0c48d68a-kube-api-access-6fxl5\") pod \"kuadrant-operator-controller-manager-86878c769f-nkw9g\" (UID: \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:36.995539 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.995507 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e131bf8b-3e21-438c-9906-fe0c0c48d68a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nkw9g\" (UID: \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:36.995757 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.995547 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fxl5\" (UniqueName: \"kubernetes.io/projected/e131bf8b-3e21-438c-9906-fe0c0c48d68a-kube-api-access-6fxl5\") pod \"kuadrant-operator-controller-manager-86878c769f-nkw9g\" (UID: \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:36.996016 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:36.995992 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e131bf8b-3e21-438c-9906-fe0c0c48d68a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nkw9g\" (UID: \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:37.003733 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:37.003709 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fxl5\" (UniqueName: \"kubernetes.io/projected/e131bf8b-3e21-438c-9906-fe0c0c48d68a-kube-api-access-6fxl5\") pod \"kuadrant-operator-controller-manager-86878c769f-nkw9g\" (UID: \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:37.108136 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:37.108035 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:37.249723 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:37.249690 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g"] Jul 10 12:53:37.251380 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:53:37.251345 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode131bf8b_3e21_438c_9906_fe0c0c48d68a.slice/crio-c4b00116134376fff9afe898cb1e01a81697d302459bd61781b90f024f851ae6 WatchSource:0}: Error finding container c4b00116134376fff9afe898cb1e01a81697d302459bd61781b90f024f851ae6: Status 404 returned error can't find the container with id c4b00116134376fff9afe898cb1e01a81697d302459bd61781b90f024f851ae6 Jul 10 12:53:37.535966 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:37.535920 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03c6eb05-1448-4c1c-a196-ae12f3f07fd1" path="/var/lib/kubelet/pods/03c6eb05-1448-4c1c-a196-ae12f3f07fd1/volumes" Jul 10 12:53:37.536372 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:37.536352 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f" path="/var/lib/kubelet/pods/7370e9fd-a2d2-47ff-8b82-4e4ec3333f7f/volumes" Jul 10 12:53:37.846573 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:37.846463 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" event={"ID":"e131bf8b-3e21-438c-9906-fe0c0c48d68a","Type":"ContainerStarted","Data":"e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30"} Jul 10 12:53:37.846573 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:37.846523 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" event={"ID":"e131bf8b-3e21-438c-9906-fe0c0c48d68a","Type":"ContainerStarted","Data":"c4b00116134376fff9afe898cb1e01a81697d302459bd61781b90f024f851ae6"} Jul 10 12:53:37.846573 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:37.846550 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:37.891839 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:37.891785 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" podStartSLOduration=1.8917710620000001 podStartE2EDuration="1.891771062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:53:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:53:37.889541284 +0000 UTC m=+468.775301902" watchObservedRunningTime="2026-07-10 12:53:37.891771062 +0000 UTC m=+468.777531678" Jul 10 12:53:47.850258 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:47.850229 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:47.850743 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:47.850311 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fx7tb" Jul 10 12:53:48.856415 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:48.856385 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 12:53:48.907976 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:48.907945 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6"] Jul 10 12:53:48.908221 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:48.908134 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" podUID="7415d6a1-6e81-46f9-b257-8f6cffecc7a9" containerName="manager" containerID="cri-o://a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5" gracePeriod=10 Jul 10 12:53:49.172157 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.172130 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:49.314590 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.314551 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5t9qn\" (UniqueName: \"kubernetes.io/projected/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-kube-api-access-5t9qn\") pod \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\" (UID: \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\") " Jul 10 12:53:49.314752 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.314603 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-extensions-socket-volume\") pod \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\" (UID: \"7415d6a1-6e81-46f9-b257-8f6cffecc7a9\") " Jul 10 12:53:49.315018 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.314999 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7415d6a1-6e81-46f9-b257-8f6cffecc7a9" (UID: "7415d6a1-6e81-46f9-b257-8f6cffecc7a9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 12:53:49.316498 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.316476 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-kube-api-access-5t9qn" (OuterVolumeSpecName: "kube-api-access-5t9qn") pod "7415d6a1-6e81-46f9-b257-8f6cffecc7a9" (UID: "7415d6a1-6e81-46f9-b257-8f6cffecc7a9"). InnerVolumeSpecName "kube-api-access-5t9qn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:53:49.415619 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.415584 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t9qn\" (UniqueName: \"kubernetes.io/projected/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-kube-api-access-5t9qn\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:53:49.415619 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.415612 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7415d6a1-6e81-46f9-b257-8f6cffecc7a9-extensions-socket-volume\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:53:49.924055 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.924016 2571 generic.go:356] "Generic (PLEG): container finished" podID="7415d6a1-6e81-46f9-b257-8f6cffecc7a9" containerID="a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5" exitCode=0 Jul 10 12:53:49.924478 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.924077 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" Jul 10 12:53:49.924478 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.924093 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" event={"ID":"7415d6a1-6e81-46f9-b257-8f6cffecc7a9","Type":"ContainerDied","Data":"a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5"} Jul 10 12:53:49.924478 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.924153 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6" event={"ID":"7415d6a1-6e81-46f9-b257-8f6cffecc7a9","Type":"ContainerDied","Data":"c278531da40d2c099afcaaafc84660addb6973432700e23802dfc3bdd54d28f2"} Jul 10 12:53:49.924478 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.924174 2571 scope.go:117] "RemoveContainer" containerID="a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5" Jul 10 12:53:49.933501 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.933472 2571 scope.go:117] "RemoveContainer" containerID="a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5" Jul 10 12:53:49.933745 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:53:49.933727 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5\": container with ID starting with a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5 not found: ID does not exist" containerID="a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5" Jul 10 12:53:49.933814 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.933759 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5"} err="failed to get container status \"a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5\": rpc error: code = NotFound desc = could not find container \"a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5\": container with ID starting with a0336e3f21662b73d1473fcd0f5f8534cb29a8985909d794688f31724e0463b5 not found: ID does not exist" Jul 10 12:53:49.955970 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.955943 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6"] Jul 10 12:53:49.974512 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:49.974485 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8pkf6"] Jul 10 12:53:51.536197 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:53:51.536164 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7415d6a1-6e81-46f9-b257-8f6cffecc7a9" path="/var/lib/kubelet/pods/7415d6a1-6e81-46f9-b257-8f6cffecc7a9/volumes" Jul 10 12:54:05.156651 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.156622 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84"] Jul 10 12:54:05.157218 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.157073 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7415d6a1-6e81-46f9-b257-8f6cffecc7a9" containerName="manager" Jul 10 12:54:05.157218 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.157086 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7415d6a1-6e81-46f9-b257-8f6cffecc7a9" containerName="manager" Jul 10 12:54:05.157218 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.157168 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7415d6a1-6e81-46f9-b257-8f6cffecc7a9" containerName="manager" Jul 10 12:54:05.164774 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.164753 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.167610 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.167576 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vf8xb\"" Jul 10 12:54:05.171057 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.171033 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84"] Jul 10 12:54:05.263448 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.263413 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.263649 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.263465 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-token\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.263649 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.263548 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-data\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.263649 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.263591 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-workload-socket\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.263649 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.263622 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c9vl\" (UniqueName: \"kubernetes.io/projected/0a12194a-bafd-421a-b2ed-7c24f8b2f292-kube-api-access-2c9vl\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.263862 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.263660 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-envoy\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.263862 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.263684 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-workload-certs\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.263862 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.263725 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-credential-socket\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.263862 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.263778 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.364713 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.364680 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.364907 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.364730 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-token\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.364907 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.364763 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-data\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.364907 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.364785 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-workload-socket\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.364907 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.364805 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9vl\" (UniqueName: \"kubernetes.io/projected/0a12194a-bafd-421a-b2ed-7c24f8b2f292-kube-api-access-2c9vl\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.364907 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.364827 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-envoy\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.364907 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.364845 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-workload-certs\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.364907 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.364873 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-credential-socket\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.364907 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.364906 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.365377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.365247 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-data\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.365377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.365264 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-workload-certs\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.365377 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.365348 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-workload-socket\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.365519 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.365483 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-credential-socket\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.365783 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.365762 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.367122 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.367103 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-envoy\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.367353 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.367336 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.374575 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.374542 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a12194a-bafd-421a-b2ed-7c24f8b2f292-istio-token\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.374575 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.374574 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c9vl\" (UniqueName: \"kubernetes.io/projected/0a12194a-bafd-421a-b2ed-7c24f8b2f292-kube-api-access-2c9vl\") pod \"maas-default-gateway-openshift-default-5df9b4f89-n5d84\" (UID: \"0a12194a-bafd-421a-b2ed-7c24f8b2f292\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.477190 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.477107 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:05.618520 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.618497 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84"] Jul 10 12:54:05.620746 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:54:05.620710 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a12194a_bafd_421a_b2ed_7c24f8b2f292.slice/crio-b4111f5860be016d265bcbf532ffe641857570adb110c2d0969c16b2eb7982ca WatchSource:0}: Error finding container b4111f5860be016d265bcbf532ffe641857570adb110c2d0969c16b2eb7982ca: Status 404 returned error can't find the container with id b4111f5860be016d265bcbf532ffe641857570adb110c2d0969c16b2eb7982ca Jul 10 12:54:05.622740 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.622698 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:54:05.622840 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.622763 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:54:05.622840 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:05.622793 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:54:06.027068 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:06.027032 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" event={"ID":"0a12194a-bafd-421a-b2ed-7c24f8b2f292","Type":"ContainerStarted","Data":"b37b7713c15949ed5a3167db94d0d92a9fa3128e02a5fb7d6b93dd750184ed3b"} Jul 10 12:54:06.027256 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:06.027095 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" event={"ID":"0a12194a-bafd-421a-b2ed-7c24f8b2f292","Type":"ContainerStarted","Data":"b4111f5860be016d265bcbf532ffe641857570adb110c2d0969c16b2eb7982ca"} Jul 10 12:54:06.477571 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:06.477535 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:06.482432 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:06.482409 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:06.504170 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:06.504128 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" podStartSLOduration=1.50411228 podStartE2EDuration="1.50411228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:54:06.052596459 +0000 UTC m=+496.938357076" watchObservedRunningTime="2026-07-10 12:54:06.50411228 +0000 UTC m=+497.389872961" Jul 10 12:54:07.033392 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:07.033353 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:07.034220 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:07.034199 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:54:08.030900 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:08.030866 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-sz7dg"] Jul 10 12:54:08.034838 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:08.034818 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sz7dg" Jul 10 12:54:08.037835 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:08.037806 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gz9pg\"" Jul 10 12:54:08.041627 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:08.041606 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sz7dg"] Jul 10 12:54:08.089548 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:08.089522 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbpkz\" (UniqueName: \"kubernetes.io/projected/cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6-kube-api-access-vbpkz\") pod \"authorino-75d556bd77-sz7dg\" (UID: \"cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6\") " pod="kuadrant-system/authorino-75d556bd77-sz7dg" Jul 10 12:54:08.190497 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:08.190465 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbpkz\" (UniqueName: \"kubernetes.io/projected/cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6-kube-api-access-vbpkz\") pod \"authorino-75d556bd77-sz7dg\" (UID: \"cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6\") " pod="kuadrant-system/authorino-75d556bd77-sz7dg" Jul 10 12:54:08.199278 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:08.199241 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbpkz\" (UniqueName: \"kubernetes.io/projected/cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6-kube-api-access-vbpkz\") pod \"authorino-75d556bd77-sz7dg\" (UID: \"cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6\") " pod="kuadrant-system/authorino-75d556bd77-sz7dg" Jul 10 12:54:08.345365 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:08.345262 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sz7dg" Jul 10 12:54:08.489904 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:08.489875 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sz7dg"] Jul 10 12:54:08.491336 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:54:08.491306 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfe1fd8b_1fc0_4e2c_9d90_c0357e9c42c6.slice/crio-7ac8c2cc7789b931ae244a121bf06f47ad0ea2a95f060ffeea66bb18c1957e7f WatchSource:0}: Error finding container 7ac8c2cc7789b931ae244a121bf06f47ad0ea2a95f060ffeea66bb18c1957e7f: Status 404 returned error can't find the container with id 7ac8c2cc7789b931ae244a121bf06f47ad0ea2a95f060ffeea66bb18c1957e7f Jul 10 12:54:09.045478 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:09.045445 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sz7dg" event={"ID":"cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6","Type":"ContainerStarted","Data":"7ac8c2cc7789b931ae244a121bf06f47ad0ea2a95f060ffeea66bb18c1957e7f"} Jul 10 12:54:12.065733 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:12.065625 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sz7dg" event={"ID":"cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6","Type":"ContainerStarted","Data":"addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f"} Jul 10 12:54:12.082517 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:12.082473 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-sz7dg" podStartSLOduration=0.812870122 podStartE2EDuration="4.082458623s" totalImagesPullingTime="3.269588501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:54:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:54:12.079980345 +0000 UTC m=+502.965740961" watchObservedRunningTime="2026-07-10 12:54:12.082458623 +0000 UTC m=+502.968219239" Jul 10 12:54:41.281191 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.281085 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f95b4f868-72z9c"] Jul 10 12:54:41.288939 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.288913 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:54:41.291117 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.291085 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f95b4f868-72z9c"] Jul 10 12:54:41.291824 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.291804 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 10 12:54:41.417102 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.417071 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ce82c0d-2268-4e79-bc69-0eead35e94ad-tls-cert\") pod \"authorino-6f95b4f868-72z9c\" (UID: \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\") " pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:54:41.417272 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.417140 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9j8x\" (UniqueName: \"kubernetes.io/projected/5ce82c0d-2268-4e79-bc69-0eead35e94ad-kube-api-access-t9j8x\") pod \"authorino-6f95b4f868-72z9c\" (UID: \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\") " pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:54:41.517666 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.517631 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9j8x\" (UniqueName: \"kubernetes.io/projected/5ce82c0d-2268-4e79-bc69-0eead35e94ad-kube-api-access-t9j8x\") pod \"authorino-6f95b4f868-72z9c\" (UID: \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\") " pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:54:41.517840 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.517711 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ce82c0d-2268-4e79-bc69-0eead35e94ad-tls-cert\") pod \"authorino-6f95b4f868-72z9c\" (UID: \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\") " pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:54:41.520224 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.520203 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ce82c0d-2268-4e79-bc69-0eead35e94ad-tls-cert\") pod \"authorino-6f95b4f868-72z9c\" (UID: \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\") " pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:54:41.527001 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.526970 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9j8x\" (UniqueName: \"kubernetes.io/projected/5ce82c0d-2268-4e79-bc69-0eead35e94ad-kube-api-access-t9j8x\") pod \"authorino-6f95b4f868-72z9c\" (UID: \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\") " pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:54:41.599353 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.599265 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:54:41.735382 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:41.735354 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f95b4f868-72z9c"] Jul 10 12:54:41.736858 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:54:41.736834 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ce82c0d_2268_4e79_bc69_0eead35e94ad.slice/crio-6cb71e58e074875195ae4174f97e27ed5f0b5832487ff43e2d9cb6ea362b0382 WatchSource:0}: Error finding container 6cb71e58e074875195ae4174f97e27ed5f0b5832487ff43e2d9cb6ea362b0382: Status 404 returned error can't find the container with id 6cb71e58e074875195ae4174f97e27ed5f0b5832487ff43e2d9cb6ea362b0382 Jul 10 12:54:42.238756 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:42.237858 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f95b4f868-72z9c" event={"ID":"5ce82c0d-2268-4e79-bc69-0eead35e94ad","Type":"ContainerStarted","Data":"67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216"} Jul 10 12:54:42.238756 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:42.237905 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f95b4f868-72z9c" event={"ID":"5ce82c0d-2268-4e79-bc69-0eead35e94ad","Type":"ContainerStarted","Data":"6cb71e58e074875195ae4174f97e27ed5f0b5832487ff43e2d9cb6ea362b0382"} Jul 10 12:54:42.256366 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:42.256308 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f95b4f868-72z9c" podStartSLOduration=0.895598067 podStartE2EDuration="1.256265196s" totalImagesPullingTime="360.667129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:54:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:54:42.252283225 +0000 UTC m=+533.138043841" watchObservedRunningTime="2026-07-10 12:54:42.256265196 +0000 UTC m=+533.142025816" Jul 10 12:54:42.275283 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:42.275248 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sz7dg"] Jul 10 12:54:42.275511 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:42.275490 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-sz7dg" podUID="cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6" containerName="authorino" containerID="cri-o://addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f" gracePeriod=30 Jul 10 12:54:42.512209 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:42.512180 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sz7dg" Jul 10 12:54:42.628582 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:42.628551 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbpkz\" (UniqueName: \"kubernetes.io/projected/cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6-kube-api-access-vbpkz\") pod \"cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6\" (UID: \"cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6\") " Jul 10 12:54:42.630526 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:42.630506 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6-kube-api-access-vbpkz" (OuterVolumeSpecName: "kube-api-access-vbpkz") pod "cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6" (UID: "cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6"). InnerVolumeSpecName "kube-api-access-vbpkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:54:42.729592 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:42.729511 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbpkz\" (UniqueName: \"kubernetes.io/projected/cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6-kube-api-access-vbpkz\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:54:43.250908 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.250874 2571 generic.go:356] "Generic (PLEG): container finished" podID="cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6" containerID="addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f" exitCode=0 Jul 10 12:54:43.251085 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.250929 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-sz7dg" Jul 10 12:54:43.251085 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.250963 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sz7dg" event={"ID":"cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6","Type":"ContainerDied","Data":"addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f"} Jul 10 12:54:43.251085 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.251006 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-sz7dg" event={"ID":"cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6","Type":"ContainerDied","Data":"7ac8c2cc7789b931ae244a121bf06f47ad0ea2a95f060ffeea66bb18c1957e7f"} Jul 10 12:54:43.251085 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.251027 2571 scope.go:117] "RemoveContainer" containerID="addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f" Jul 10 12:54:43.260406 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.260390 2571 scope.go:117] "RemoveContainer" containerID="addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f" Jul 10 12:54:43.260645 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:54:43.260627 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f\": container with ID starting with addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f not found: ID does not exist" containerID="addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f" Jul 10 12:54:43.260693 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.260655 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f"} err="failed to get container status \"addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f\": rpc error: code = NotFound desc = could not find container \"addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f\": container with ID starting with addef22b43c9e7a2c0fbb43895c32e4a65cfbd9f05435c2a57d6bcf8a425921f not found: ID does not exist" Jul 10 12:54:43.271874 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.271850 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sz7dg"] Jul 10 12:54:43.274755 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.274733 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-sz7dg"] Jul 10 12:54:43.536581 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:54:43.536502 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6" path="/var/lib/kubelet/pods/cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6/volumes" Jul 10 12:55:22.667533 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.667493 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-f92z7"] Jul 10 12:55:22.668165 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.668146 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6" containerName="authorino" Jul 10 12:55:22.668256 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.668169 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6" containerName="authorino" Jul 10 12:55:22.668331 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.668316 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfe1fd8b-1fc0-4e2c-9d90-c0357e9c42c6" containerName="authorino" Jul 10 12:55:22.671579 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.671557 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:22.674160 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.674141 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wkjqj\"" Jul 10 12:55:22.674260 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.674210 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 12:55:22.683577 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.683551 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-f92z7"] Jul 10 12:55:22.798846 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.798808 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qcqw\" (UniqueName: \"kubernetes.io/projected/4773c3e1-c9ef-489a-bc27-7843f5aec186-kube-api-access-9qcqw\") pod \"payload-pre-processing-7964457696-f92z7\" (UID: \"4773c3e1-c9ef-489a-bc27-7843f5aec186\") " pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:22.799035 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.798852 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4773c3e1-c9ef-489a-bc27-7843f5aec186-plugins-config-volume\") pod \"payload-pre-processing-7964457696-f92z7\" (UID: \"4773c3e1-c9ef-489a-bc27-7843f5aec186\") " pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:22.899993 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.899965 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qcqw\" (UniqueName: \"kubernetes.io/projected/4773c3e1-c9ef-489a-bc27-7843f5aec186-kube-api-access-9qcqw\") pod \"payload-pre-processing-7964457696-f92z7\" (UID: \"4773c3e1-c9ef-489a-bc27-7843f5aec186\") " pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:22.900148 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.899996 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4773c3e1-c9ef-489a-bc27-7843f5aec186-plugins-config-volume\") pod \"payload-pre-processing-7964457696-f92z7\" (UID: \"4773c3e1-c9ef-489a-bc27-7843f5aec186\") " pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:22.900671 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.900653 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4773c3e1-c9ef-489a-bc27-7843f5aec186-plugins-config-volume\") pod \"payload-pre-processing-7964457696-f92z7\" (UID: \"4773c3e1-c9ef-489a-bc27-7843f5aec186\") " pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:22.908620 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.908595 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qcqw\" (UniqueName: \"kubernetes.io/projected/4773c3e1-c9ef-489a-bc27-7843f5aec186-kube-api-access-9qcqw\") pod \"payload-pre-processing-7964457696-f92z7\" (UID: \"4773c3e1-c9ef-489a-bc27-7843f5aec186\") " pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:22.983797 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:22.983710 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:23.123900 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:23.123875 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-f92z7"] Jul 10 12:55:23.126379 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:55:23.126354 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4773c3e1_c9ef_489a_bc27_7843f5aec186.slice/crio-6906c100a3acfe845989be368750da3da69f65a6fa6d32e75da10726adf9a0e9 WatchSource:0}: Error finding container 6906c100a3acfe845989be368750da3da69f65a6fa6d32e75da10726adf9a0e9: Status 404 returned error can't find the container with id 6906c100a3acfe845989be368750da3da69f65a6fa6d32e75da10726adf9a0e9 Jul 10 12:55:23.491360 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:23.491325 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" event={"ID":"4773c3e1-c9ef-489a-bc27-7843f5aec186","Type":"ContainerStarted","Data":"6906c100a3acfe845989be368750da3da69f65a6fa6d32e75da10726adf9a0e9"} Jul 10 12:55:25.505330 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:25.505264 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" event={"ID":"4773c3e1-c9ef-489a-bc27-7843f5aec186","Type":"ContainerStarted","Data":"86fc52fa5660b8afffa36a1d1baf4f3695d1fb6cee7e9ee4be1722c7114d76c9"} Jul 10 12:55:25.505743 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:25.505479 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:25.522016 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:25.521952 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" podStartSLOduration=1.903740733 podStartE2EDuration="3.521934121s" totalImagesPullingTime="1.618193388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:55:25.521260872 +0000 UTC m=+576.407021479" watchObservedRunningTime="2026-07-10 12:55:25.521934121 +0000 UTC m=+576.407694737" Jul 10 12:55:30.549566 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:30.549472 2571 generic.go:356] "Generic (PLEG): container finished" podID="0a12194a-bafd-421a-b2ed-7c24f8b2f292" containerID="b37b7713c15949ed5a3167db94d0d92a9fa3128e02a5fb7d6b93dd750184ed3b" exitCode=0 Jul 10 12:55:30.549566 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:30.549546 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" event={"ID":"0a12194a-bafd-421a-b2ed-7c24f8b2f292","Type":"ContainerDied","Data":"b37b7713c15949ed5a3167db94d0d92a9fa3128e02a5fb7d6b93dd750184ed3b"} Jul 10 12:55:30.550050 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:30.549918 2571 scope.go:117] "RemoveContainer" containerID="b37b7713c15949ed5a3167db94d0d92a9fa3128e02a5fb7d6b93dd750184ed3b" Jul 10 12:55:30.550896 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:30.550863 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:55:30.551009 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:30.550927 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:55:30.551009 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:30.550959 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 12:55:31.557342 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:31.557280 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" event={"ID":"0a12194a-bafd-421a-b2ed-7c24f8b2f292","Type":"ContainerStarted","Data":"0d273b20fc46eb678df953ffd24cbf0d39cc79ee697271b0aae8ce8cd6b54638"} Jul 10 12:55:32.477489 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:32.477451 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:55:32.478698 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:32.478675 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5df9b4f89-n5d84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 12:55:32.478759 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:32.478725 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" podUID="0a12194a-bafd-421a-b2ed-7c24f8b2f292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 12:55:33.477783 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:33.477748 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5df9b4f89-n5d84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 12:55:33.478157 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:33.477801 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" podUID="0a12194a-bafd-421a-b2ed-7c24f8b2f292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 12:55:34.478405 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:34.478374 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5df9b4f89-n5d84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 12:55:34.478785 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:34.478430 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" podUID="0a12194a-bafd-421a-b2ed-7c24f8b2f292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 12:55:35.477936 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:35.477893 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5df9b4f89-n5d84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 10 12:55:35.478112 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:35.477967 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" podUID="0a12194a-bafd-421a-b2ed-7c24f8b2f292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 10 12:55:36.481347 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:36.481319 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:55:36.481736 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:36.481577 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:55:36.482174 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:36.482157 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5df9b4f89-n5d84" Jul 10 12:55:36.513814 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:36.513791 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-f92z7" Jul 10 12:55:39.066405 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.066374 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98"] Jul 10 12:55:39.072719 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.072700 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.075448 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.075427 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 12:55:39.076442 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.076423 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 10 12:55:39.076673 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.076583 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mkb4s\"" Jul 10 12:55:39.076771 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.076655 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 12:55:39.078580 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.078558 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98"] Jul 10 12:55:39.170096 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.170066 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc2jb\" (UniqueName: \"kubernetes.io/projected/6edb48aa-5ea8-423a-9c1c-477d762bb07d-kube-api-access-nc2jb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.170248 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.170130 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.170248 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.170194 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.170248 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.170240 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.170450 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.170283 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.170450 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.170343 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6edb48aa-5ea8-423a-9c1c-477d762bb07d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.270976 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.270948 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.271151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.270994 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.271151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.271024 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.271151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.271062 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.271151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.271081 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6edb48aa-5ea8-423a-9c1c-477d762bb07d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.271151 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.271135 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc2jb\" (UniqueName: \"kubernetes.io/projected/6edb48aa-5ea8-423a-9c1c-477d762bb07d-kube-api-access-nc2jb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.271444 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.271403 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.271495 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.271478 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.271532 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.271513 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.273210 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.273190 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6edb48aa-5ea8-423a-9c1c-477d762bb07d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.273508 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.273490 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6edb48aa-5ea8-423a-9c1c-477d762bb07d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.279325 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.279305 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc2jb\" (UniqueName: \"kubernetes.io/projected/6edb48aa-5ea8-423a-9c1c-477d762bb07d-kube-api-access-nc2jb\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98\" (UID: \"6edb48aa-5ea8-423a-9c1c-477d762bb07d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.386075 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.385993 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:39.527110 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.527082 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98"] Jul 10 12:55:39.604797 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:39.604762 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" event={"ID":"6edb48aa-5ea8-423a-9c1c-477d762bb07d","Type":"ContainerStarted","Data":"677f9512e25baa5af08362753e51bc2ed43133c616657f560b03f4d5160a17c0"} Jul 10 12:55:41.618027 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:41.617994 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" event={"ID":"6edb48aa-5ea8-423a-9c1c-477d762bb07d","Type":"ContainerStarted","Data":"75ca469f9d3a8e2dcf97062ea44ad26779dc20f3b8aa7bcc376fd18ce07cb30e"} Jul 10 12:55:41.636724 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:41.636670 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" podStartSLOduration=1.374435185 podStartE2EDuration="2.636655795s" totalImagesPullingTime="1.26222061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:55:41.633832917 +0000 UTC m=+592.519593545" watchObservedRunningTime="2026-07-10 12:55:41.636655795 +0000 UTC m=+592.522416410" Jul 10 12:55:49.386310 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:49.386260 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:49.386696 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:49.386341 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:49.398926 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:49.398900 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:55:49.675875 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:55:49.675849 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98" Jul 10 12:57:41.121078 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.121003 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56986cdcfc-tvxh8"] Jul 10 12:57:41.124864 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.124843 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56986cdcfc-tvxh8" Jul 10 12:57:41.133968 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.133939 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56986cdcfc-tvxh8"] Jul 10 12:57:41.186267 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.186236 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwb8s\" (UniqueName: \"kubernetes.io/projected/cdc8816a-0460-4f96-80e7-78bf79157b23-kube-api-access-dwb8s\") pod \"authorino-56986cdcfc-tvxh8\" (UID: \"cdc8816a-0460-4f96-80e7-78bf79157b23\") " pod="kuadrant-system/authorino-56986cdcfc-tvxh8" Jul 10 12:57:41.186456 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.186280 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cdc8816a-0460-4f96-80e7-78bf79157b23-tls-cert\") pod \"authorino-56986cdcfc-tvxh8\" (UID: \"cdc8816a-0460-4f96-80e7-78bf79157b23\") " pod="kuadrant-system/authorino-56986cdcfc-tvxh8" Jul 10 12:57:41.287633 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.287600 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwb8s\" (UniqueName: \"kubernetes.io/projected/cdc8816a-0460-4f96-80e7-78bf79157b23-kube-api-access-dwb8s\") pod \"authorino-56986cdcfc-tvxh8\" (UID: \"cdc8816a-0460-4f96-80e7-78bf79157b23\") " pod="kuadrant-system/authorino-56986cdcfc-tvxh8" Jul 10 12:57:41.287769 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.287645 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cdc8816a-0460-4f96-80e7-78bf79157b23-tls-cert\") pod \"authorino-56986cdcfc-tvxh8\" (UID: \"cdc8816a-0460-4f96-80e7-78bf79157b23\") " pod="kuadrant-system/authorino-56986cdcfc-tvxh8" Jul 10 12:57:41.290008 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.289990 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/cdc8816a-0460-4f96-80e7-78bf79157b23-tls-cert\") pod \"authorino-56986cdcfc-tvxh8\" (UID: \"cdc8816a-0460-4f96-80e7-78bf79157b23\") " pod="kuadrant-system/authorino-56986cdcfc-tvxh8" Jul 10 12:57:41.295628 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.295605 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwb8s\" (UniqueName: \"kubernetes.io/projected/cdc8816a-0460-4f96-80e7-78bf79157b23-kube-api-access-dwb8s\") pod \"authorino-56986cdcfc-tvxh8\" (UID: \"cdc8816a-0460-4f96-80e7-78bf79157b23\") " pod="kuadrant-system/authorino-56986cdcfc-tvxh8" Jul 10 12:57:41.434631 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.434602 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56986cdcfc-tvxh8" Jul 10 12:57:41.575575 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.575552 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56986cdcfc-tvxh8"] Jul 10 12:57:41.577470 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:57:41.577442 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdc8816a_0460_4f96_80e7_78bf79157b23.slice/crio-0f409ef29006b7dcc5c39c747d5662f9823c7389671de591710818db17780cef WatchSource:0}: Error finding container 0f409ef29006b7dcc5c39c747d5662f9823c7389671de591710818db17780cef: Status 404 returned error can't find the container with id 0f409ef29006b7dcc5c39c747d5662f9823c7389671de591710818db17780cef Jul 10 12:57:41.579107 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:41.579089 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 12:57:42.332284 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.332198 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56986cdcfc-tvxh8" event={"ID":"cdc8816a-0460-4f96-80e7-78bf79157b23","Type":"ContainerStarted","Data":"a136a1d8b5b82468840f95525e4206ce91495483c8277c037fa89e742e3a3e2a"} Jul 10 12:57:42.332757 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.332738 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56986cdcfc-tvxh8" event={"ID":"cdc8816a-0460-4f96-80e7-78bf79157b23","Type":"ContainerStarted","Data":"0f409ef29006b7dcc5c39c747d5662f9823c7389671de591710818db17780cef"} Jul 10 12:57:42.352168 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.352117 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56986cdcfc-tvxh8" podStartSLOduration=0.904239384 podStartE2EDuration="1.352101681s" totalImagesPullingTime="447.862297ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:57:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:57:42.34846891 +0000 UTC m=+713.234229541" watchObservedRunningTime="2026-07-10 12:57:42.352101681 +0000 UTC m=+713.237862295" Jul 10 12:57:42.383477 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.383445 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f95b4f868-72z9c"] Jul 10 12:57:42.383681 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.383653 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6f95b4f868-72z9c" podUID="5ce82c0d-2268-4e79-bc69-0eead35e94ad" containerName="authorino" containerID="cri-o://67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216" gracePeriod=30 Jul 10 12:57:42.665881 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.665856 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:57:42.703141 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.703105 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9j8x\" (UniqueName: \"kubernetes.io/projected/5ce82c0d-2268-4e79-bc69-0eead35e94ad-kube-api-access-t9j8x\") pod \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\" (UID: \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\") " Jul 10 12:57:42.703371 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.703243 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ce82c0d-2268-4e79-bc69-0eead35e94ad-tls-cert\") pod \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\" (UID: \"5ce82c0d-2268-4e79-bc69-0eead35e94ad\") " Jul 10 12:57:42.705455 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.705422 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce82c0d-2268-4e79-bc69-0eead35e94ad-kube-api-access-t9j8x" (OuterVolumeSpecName: "kube-api-access-t9j8x") pod "5ce82c0d-2268-4e79-bc69-0eead35e94ad" (UID: "5ce82c0d-2268-4e79-bc69-0eead35e94ad"). InnerVolumeSpecName "kube-api-access-t9j8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:57:42.714781 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.714758 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ce82c0d-2268-4e79-bc69-0eead35e94ad-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5ce82c0d-2268-4e79-bc69-0eead35e94ad" (UID: "5ce82c0d-2268-4e79-bc69-0eead35e94ad"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:57:42.804818 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.804786 2571 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5ce82c0d-2268-4e79-bc69-0eead35e94ad-tls-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:57:42.804818 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:42.804814 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9j8x\" (UniqueName: \"kubernetes.io/projected/5ce82c0d-2268-4e79-bc69-0eead35e94ad-kube-api-access-t9j8x\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:57:43.341609 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.341570 2571 generic.go:356] "Generic (PLEG): container finished" podID="5ce82c0d-2268-4e79-bc69-0eead35e94ad" containerID="67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216" exitCode=0 Jul 10 12:57:43.342075 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.341620 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f95b4f868-72z9c" Jul 10 12:57:43.342075 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.341651 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f95b4f868-72z9c" event={"ID":"5ce82c0d-2268-4e79-bc69-0eead35e94ad","Type":"ContainerDied","Data":"67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216"} Jul 10 12:57:43.342075 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.341708 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f95b4f868-72z9c" event={"ID":"5ce82c0d-2268-4e79-bc69-0eead35e94ad","Type":"ContainerDied","Data":"6cb71e58e074875195ae4174f97e27ed5f0b5832487ff43e2d9cb6ea362b0382"} Jul 10 12:57:43.342075 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.341734 2571 scope.go:117] "RemoveContainer" containerID="67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216" Jul 10 12:57:43.353418 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.353368 2571 scope.go:117] "RemoveContainer" containerID="67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216" Jul 10 12:57:43.353704 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:57:43.353683 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216\": container with ID starting with 67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216 not found: ID does not exist" containerID="67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216" Jul 10 12:57:43.353799 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.353710 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216"} err="failed to get container status \"67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216\": rpc error: code = NotFound desc = could not find container \"67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216\": container with ID starting with 67e0fce3fc372aaffb283e405b38006c6770ddeb9a6eabad20e829ce9b114216 not found: ID does not exist" Jul 10 12:57:43.363684 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.363658 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f95b4f868-72z9c"] Jul 10 12:57:43.366059 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.366034 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f95b4f868-72z9c"] Jul 10 12:57:43.535838 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:57:43.535809 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ce82c0d-2268-4e79-bc69-0eead35e94ad" path="/var/lib/kubelet/pods/5ce82c0d-2268-4e79-bc69-0eead35e94ad/volumes" Jul 10 12:58:18.925327 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:18.925270 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78864954cb-gj78d"] Jul 10 12:58:18.925955 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:18.925936 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ce82c0d-2268-4e79-bc69-0eead35e94ad" containerName="authorino" Jul 10 12:58:18.925955 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:18.925956 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ce82c0d-2268-4e79-bc69-0eead35e94ad" containerName="authorino" Jul 10 12:58:18.926078 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:18.926027 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ce82c0d-2268-4e79-bc69-0eead35e94ad" containerName="authorino" Jul 10 12:58:18.929310 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:18.929273 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:18.940770 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:18.940745 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78864954cb-gj78d"] Jul 10 12:58:19.057470 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.057436 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-service-ca\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.057652 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.057476 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-console-oauth-config\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.057652 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.057521 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-console-config\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.057652 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.057562 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-oauth-serving-cert\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.057652 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.057584 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5dg2\" (UniqueName: \"kubernetes.io/projected/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-kube-api-access-c5dg2\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.057652 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.057600 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-trusted-ca-bundle\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.057652 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.057624 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-console-serving-cert\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.158234 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.158201 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-console-oauth-config\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.158427 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.158252 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-console-config\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.158427 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.158279 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-oauth-serving-cert\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.158427 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.158324 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5dg2\" (UniqueName: \"kubernetes.io/projected/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-kube-api-access-c5dg2\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.158427 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.158341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-trusted-ca-bundle\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.158427 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.158378 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-console-serving-cert\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.158427 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.158428 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-service-ca\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.159049 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.159021 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-console-config\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.159217 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.159172 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-oauth-serving-cert\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.159217 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.159178 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-service-ca\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.159327 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.159256 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-trusted-ca-bundle\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.160856 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.160834 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-console-serving-cert\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.160960 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.160940 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-console-oauth-config\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.166487 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.166457 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5dg2\" (UniqueName: \"kubernetes.io/projected/4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35-kube-api-access-c5dg2\") pod \"console-78864954cb-gj78d\" (UID: \"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35\") " pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.240103 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.240040 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:19.380650 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.380620 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78864954cb-gj78d"] Jul 10 12:58:19.382039 ip-10-0-143-162 kubenswrapper[2571]: W0710 12:58:19.382009 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e1ead29_c08c_4f7c_b2b6_b5dcc1bcbf35.slice/crio-a5d9034405fa7b0c9491c6e96647e2ffc208284540d5fc72d50b96482eea70ef WatchSource:0}: Error finding container a5d9034405fa7b0c9491c6e96647e2ffc208284540d5fc72d50b96482eea70ef: Status 404 returned error can't find the container with id a5d9034405fa7b0c9491c6e96647e2ffc208284540d5fc72d50b96482eea70ef Jul 10 12:58:19.561631 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.561549 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78864954cb-gj78d" event={"ID":"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35","Type":"ContainerStarted","Data":"98cad8102a18fc1a994cfe0fd93bd0db8abedb13f088c4dcdfad1c9dc11236ac"} Jul 10 12:58:19.561631 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.561602 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78864954cb-gj78d" event={"ID":"4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35","Type":"ContainerStarted","Data":"a5d9034405fa7b0c9491c6e96647e2ffc208284540d5fc72d50b96482eea70ef"} Jul 10 12:58:19.580942 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:19.580897 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78864954cb-gj78d" podStartSLOduration=1.580882594 podStartE2EDuration="1.580882594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 12:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 12:58:19.578822006 +0000 UTC m=+750.464582622" watchObservedRunningTime="2026-07-10 12:58:19.580882594 +0000 UTC m=+750.466643209" Jul 10 12:58:29.240454 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:29.240416 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:29.240957 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:29.240505 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:29.245352 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:29.245330 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:29.628680 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:29.628597 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78864954cb-gj78d" Jul 10 12:58:29.677658 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:29.677622 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b8c6f76b-dr2d5"] Jul 10 12:58:54.703382 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:54.703317 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77b8c6f76b-dr2d5" podUID="524ef394-21a4-4bef-bdec-c3728d7a94b3" containerName="console" containerID="cri-o://542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969" gracePeriod=15 Jul 10 12:58:54.969939 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:54.969916 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:58:55.104631 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.104599 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfc8t\" (UniqueName: \"kubernetes.io/projected/524ef394-21a4-4bef-bdec-c3728d7a94b3-kube-api-access-bfc8t\") pod \"524ef394-21a4-4bef-bdec-c3728d7a94b3\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " Jul 10 12:58:55.104829 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.104657 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-oauth-serving-cert\") pod \"524ef394-21a4-4bef-bdec-c3728d7a94b3\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " Jul 10 12:58:55.104829 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.104679 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-oauth-config\") pod \"524ef394-21a4-4bef-bdec-c3728d7a94b3\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " Jul 10 12:58:55.104829 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.104781 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-service-ca\") pod \"524ef394-21a4-4bef-bdec-c3728d7a94b3\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " Jul 10 12:58:55.104829 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.104822 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-trusted-ca-bundle\") pod \"524ef394-21a4-4bef-bdec-c3728d7a94b3\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " Jul 10 12:58:55.105053 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.104873 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-config\") pod \"524ef394-21a4-4bef-bdec-c3728d7a94b3\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " Jul 10 12:58:55.105053 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.104949 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-serving-cert\") pod \"524ef394-21a4-4bef-bdec-c3728d7a94b3\" (UID: \"524ef394-21a4-4bef-bdec-c3728d7a94b3\") " Jul 10 12:58:55.105201 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.105160 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-service-ca" (OuterVolumeSpecName: "service-ca") pod "524ef394-21a4-4bef-bdec-c3728d7a94b3" (UID: "524ef394-21a4-4bef-bdec-c3728d7a94b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:58:55.105201 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.105176 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "524ef394-21a4-4bef-bdec-c3728d7a94b3" (UID: "524ef394-21a4-4bef-bdec-c3728d7a94b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:58:55.105338 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.105233 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-config" (OuterVolumeSpecName: "console-config") pod "524ef394-21a4-4bef-bdec-c3728d7a94b3" (UID: "524ef394-21a4-4bef-bdec-c3728d7a94b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:58:55.105389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.105333 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-oauth-serving-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:58:55.105389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.105334 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "524ef394-21a4-4bef-bdec-c3728d7a94b3" (UID: "524ef394-21a4-4bef-bdec-c3728d7a94b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 12:58:55.105389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.105351 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-service-ca\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:58:55.105389 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.105365 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-config\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:58:55.107100 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.107076 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "524ef394-21a4-4bef-bdec-c3728d7a94b3" (UID: "524ef394-21a4-4bef-bdec-c3728d7a94b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:58:55.107316 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.107267 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "524ef394-21a4-4bef-bdec-c3728d7a94b3" (UID: "524ef394-21a4-4bef-bdec-c3728d7a94b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 12:58:55.107413 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.107320 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/524ef394-21a4-4bef-bdec-c3728d7a94b3-kube-api-access-bfc8t" (OuterVolumeSpecName: "kube-api-access-bfc8t") pod "524ef394-21a4-4bef-bdec-c3728d7a94b3" (UID: "524ef394-21a4-4bef-bdec-c3728d7a94b3"). InnerVolumeSpecName "kube-api-access-bfc8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 12:58:55.206794 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.206728 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfc8t\" (UniqueName: \"kubernetes.io/projected/524ef394-21a4-4bef-bdec-c3728d7a94b3-kube-api-access-bfc8t\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:58:55.206794 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.206751 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-oauth-config\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:58:55.206794 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.206759 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/524ef394-21a4-4bef-bdec-c3728d7a94b3-trusted-ca-bundle\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:58:55.206794 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.206768 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/524ef394-21a4-4bef-bdec-c3728d7a94b3-console-serving-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 12:58:55.790922 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.790891 2571 generic.go:356] "Generic (PLEG): container finished" podID="524ef394-21a4-4bef-bdec-c3728d7a94b3" containerID="542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969" exitCode=0 Jul 10 12:58:55.791337 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.790933 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b8c6f76b-dr2d5" Jul 10 12:58:55.791337 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.790977 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b8c6f76b-dr2d5" event={"ID":"524ef394-21a4-4bef-bdec-c3728d7a94b3","Type":"ContainerDied","Data":"542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969"} Jul 10 12:58:55.791337 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.791025 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b8c6f76b-dr2d5" event={"ID":"524ef394-21a4-4bef-bdec-c3728d7a94b3","Type":"ContainerDied","Data":"5bcd80d4cf6ed63eac56034c8944dd1568f5cd70307e8d95e5520f0e6584ae65"} Jul 10 12:58:55.791337 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.791049 2571 scope.go:117] "RemoveContainer" containerID="542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969" Jul 10 12:58:55.800475 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.800445 2571 scope.go:117] "RemoveContainer" containerID="542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969" Jul 10 12:58:55.800726 ip-10-0-143-162 kubenswrapper[2571]: E0710 12:58:55.800709 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969\": container with ID starting with 542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969 not found: ID does not exist" containerID="542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969" Jul 10 12:58:55.800776 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.800733 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969"} err="failed to get container status \"542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969\": rpc error: code = NotFound desc = could not find container \"542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969\": container with ID starting with 542800fdfb834471fadbc43aee98ccdf2171d685ecb971871e9ec7ec37eed969 not found: ID does not exist" Jul 10 12:58:55.810540 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.810516 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b8c6f76b-dr2d5"] Jul 10 12:58:55.812935 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:55.812916 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b8c6f76b-dr2d5"] Jul 10 12:58:57.535806 ip-10-0-143-162 kubenswrapper[2571]: I0710 12:58:57.535771 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="524ef394-21a4-4bef-bdec-c3728d7a94b3" path="/var/lib/kubelet/pods/524ef394-21a4-4bef-bdec-c3728d7a94b3/volumes" Jul 10 13:00:00.136835 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:00.136799 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9"] Jul 10 13:00:00.137274 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:00.137259 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="524ef394-21a4-4bef-bdec-c3728d7a94b3" containerName="console" Jul 10 13:00:00.137274 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:00.137275 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="524ef394-21a4-4bef-bdec-c3728d7a94b3" containerName="console" Jul 10 13:00:00.137382 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:00.137363 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="524ef394-21a4-4bef-bdec-c3728d7a94b3" containerName="console" Jul 10 13:00:00.141627 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:00.141612 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" Jul 10 13:00:00.144564 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:00.144540 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sfczk\"" Jul 10 13:00:00.147307 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:00.147263 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9"] Jul 10 13:00:00.153028 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:00.153010 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" Jul 10 13:00:00.498777 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:00.498750 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9"] Jul 10 13:00:00.500137 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:00:00.500111 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e35d548_dfd0_4919_bcd3_c0d56878033f.slice/crio-32b123e35e306c7a7a33308e7d4a02c2c921fb6fd971442853baa0911a3b76fe WatchSource:0}: Error finding container 32b123e35e306c7a7a33308e7d4a02c2c921fb6fd971442853baa0911a3b76fe: Status 404 returned error can't find the container with id 32b123e35e306c7a7a33308e7d4a02c2c921fb6fd971442853baa0911a3b76fe Jul 10 13:00:01.181776 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:01.181740 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" event={"ID":"3e35d548-dfd0-4919-bcd3-c0d56878033f","Type":"ContainerStarted","Data":"32b123e35e306c7a7a33308e7d4a02c2c921fb6fd971442853baa0911a3b76fe"} Jul 10 13:00:20.298558 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:20.298455 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" event={"ID":"3e35d548-dfd0-4919-bcd3-c0d56878033f","Type":"ContainerStarted","Data":"0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c"} Jul 10 13:00:20.312797 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:00:20.312744 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" podStartSLOduration=1.167917207 podStartE2EDuration="20.312727122s" totalImagesPullingTime="19.144809915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:00:20.312222736 +0000 UTC m=+871.197983367" watchObservedRunningTime="2026-07-10 13:00:20.312727122 +0000 UTC m=+871.198487739" Jul 10 13:01:00.011113 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:00.011081 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9"] Jul 10 13:01:00.011588 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:00.011272 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" podUID="3e35d548-dfd0-4919-bcd3-c0d56878033f" containerName="cleanup" containerID="cri-o://0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c" gracePeriod=30 Jul 10 13:01:30.206046 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.206019 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" Jul 10 13:01:30.726987 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.726949 2571 generic.go:356] "Generic (PLEG): container finished" podID="3e35d548-dfd0-4919-bcd3-c0d56878033f" containerID="0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c" exitCode=137 Jul 10 13:01:30.727176 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.727012 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" Jul 10 13:01:30.727176 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.727042 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" event={"ID":"3e35d548-dfd0-4919-bcd3-c0d56878033f","Type":"ContainerDied","Data":"0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c"} Jul 10 13:01:30.727176 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.727099 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9" event={"ID":"3e35d548-dfd0-4919-bcd3-c0d56878033f","Type":"ContainerDied","Data":"32b123e35e306c7a7a33308e7d4a02c2c921fb6fd971442853baa0911a3b76fe"} Jul 10 13:01:30.727176 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.727120 2571 scope.go:117] "RemoveContainer" containerID="0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c" Jul 10 13:01:30.736495 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.736472 2571 scope.go:117] "RemoveContainer" containerID="0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c" Jul 10 13:01:30.736744 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:01:30.736726 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c\": container with ID starting with 0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c not found: ID does not exist" containerID="0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c" Jul 10 13:01:30.736792 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.736752 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c"} err="failed to get container status \"0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c\": rpc error: code = NotFound desc = could not find container \"0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c\": container with ID starting with 0fd342867076ae3ea4e6a45b8223e4acd3bd05ccf5d59ce82b7278badf64094c not found: ID does not exist" Jul 10 13:01:30.750821 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.750787 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9"] Jul 10 13:01:30.754997 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:30.754980 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728140-kf5h9"] Jul 10 13:01:31.535472 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:01:31.535440 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e35d548-dfd0-4919-bcd3-c0d56878033f" path="/var/lib/kubelet/pods/3e35d548-dfd0-4919-bcd3-c0d56878033f/volumes" Jul 10 13:05:28.452788 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:28.452704 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g"] Jul 10 13:05:28.453339 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:28.452890 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" podUID="e131bf8b-3e21-438c-9906-fe0c0c48d68a" containerName="manager" containerID="cri-o://e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30" gracePeriod=10 Jul 10 13:05:30.823687 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:30.823660 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 13:05:30.953829 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:30.953799 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e131bf8b-3e21-438c-9906-fe0c0c48d68a-extensions-socket-volume\") pod \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\" (UID: \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\") " Jul 10 13:05:30.954008 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:30.953902 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fxl5\" (UniqueName: \"kubernetes.io/projected/e131bf8b-3e21-438c-9906-fe0c0c48d68a-kube-api-access-6fxl5\") pod \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\" (UID: \"e131bf8b-3e21-438c-9906-fe0c0c48d68a\") " Jul 10 13:05:30.954146 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:30.954110 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e131bf8b-3e21-438c-9906-fe0c0c48d68a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e131bf8b-3e21-438c-9906-fe0c0c48d68a" (UID: "e131bf8b-3e21-438c-9906-fe0c0c48d68a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:05:30.956132 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:30.956105 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e131bf8b-3e21-438c-9906-fe0c0c48d68a-kube-api-access-6fxl5" (OuterVolumeSpecName: "kube-api-access-6fxl5") pod "e131bf8b-3e21-438c-9906-fe0c0c48d68a" (UID: "e131bf8b-3e21-438c-9906-fe0c0c48d68a"). InnerVolumeSpecName "kube-api-access-6fxl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:05:31.054774 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.054748 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fxl5\" (UniqueName: \"kubernetes.io/projected/e131bf8b-3e21-438c-9906-fe0c0c48d68a-kube-api-access-6fxl5\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:05:31.054774 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.054772 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e131bf8b-3e21-438c-9906-fe0c0c48d68a-extensions-socket-volume\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:05:31.153153 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.153122 2571 generic.go:356] "Generic (PLEG): container finished" podID="e131bf8b-3e21-438c-9906-fe0c0c48d68a" containerID="e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30" exitCode=0 Jul 10 13:05:31.153250 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.153163 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" event={"ID":"e131bf8b-3e21-438c-9906-fe0c0c48d68a","Type":"ContainerDied","Data":"e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30"} Jul 10 13:05:31.153250 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.153174 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" Jul 10 13:05:31.153250 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.153214 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g" event={"ID":"e131bf8b-3e21-438c-9906-fe0c0c48d68a","Type":"ContainerDied","Data":"c4b00116134376fff9afe898cb1e01a81697d302459bd61781b90f024f851ae6"} Jul 10 13:05:31.153250 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.153243 2571 scope.go:117] "RemoveContainer" containerID="e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30" Jul 10 13:05:31.163716 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.163700 2571 scope.go:117] "RemoveContainer" containerID="e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30" Jul 10 13:05:31.163949 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:05:31.163933 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30\": container with ID starting with e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30 not found: ID does not exist" containerID="e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30" Jul 10 13:05:31.163993 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.163957 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30"} err="failed to get container status \"e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30\": rpc error: code = NotFound desc = could not find container \"e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30\": container with ID starting with e2574191d6a3aa54884f5dc66030ce46bed23c81d864be57a1d17b8d12b8fa30 not found: ID does not exist" Jul 10 13:05:31.176579 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.176559 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g"] Jul 10 13:05:31.179849 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.179832 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nkw9g"] Jul 10 13:05:31.535255 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:05:31.535225 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e131bf8b-3e21-438c-9906-fe0c0c48d68a" path="/var/lib/kubelet/pods/e131bf8b-3e21-438c-9906-fe0c0c48d68a/volumes" Jul 10 13:06:29.554132 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.554102 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69"] Jul 10 13:06:29.554838 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.554530 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e131bf8b-3e21-438c-9906-fe0c0c48d68a" containerName="manager" Jul 10 13:06:29.554838 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.554542 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e131bf8b-3e21-438c-9906-fe0c0c48d68a" containerName="manager" Jul 10 13:06:29.554838 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.554558 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e35d548-dfd0-4919-bcd3-c0d56878033f" containerName="cleanup" Jul 10 13:06:29.554838 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.554564 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e35d548-dfd0-4919-bcd3-c0d56878033f" containerName="cleanup" Jul 10 13:06:29.554838 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.554655 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e35d548-dfd0-4919-bcd3-c0d56878033f" containerName="cleanup" Jul 10 13:06:29.554838 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.554663 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e131bf8b-3e21-438c-9906-fe0c0c48d68a" containerName="manager" Jul 10 13:06:29.556953 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.556935 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:06:29.559730 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.559703 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8z5lv\"" Jul 10 13:06:29.572207 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.572185 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69"] Jul 10 13:06:29.672615 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.672583 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e912041a-26ec-4ce5-8bc3-84f54997b916-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zvz69\" (UID: \"e912041a-26ec-4ce5-8bc3-84f54997b916\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:06:29.672773 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.672752 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsdhr\" (UniqueName: \"kubernetes.io/projected/e912041a-26ec-4ce5-8bc3-84f54997b916-kube-api-access-nsdhr\") pod \"kuadrant-operator-controller-manager-86878c769f-zvz69\" (UID: \"e912041a-26ec-4ce5-8bc3-84f54997b916\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:06:29.773442 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.773411 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsdhr\" (UniqueName: \"kubernetes.io/projected/e912041a-26ec-4ce5-8bc3-84f54997b916-kube-api-access-nsdhr\") pod \"kuadrant-operator-controller-manager-86878c769f-zvz69\" (UID: \"e912041a-26ec-4ce5-8bc3-84f54997b916\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:06:29.773608 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.773462 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e912041a-26ec-4ce5-8bc3-84f54997b916-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zvz69\" (UID: \"e912041a-26ec-4ce5-8bc3-84f54997b916\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:06:29.773853 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.773834 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e912041a-26ec-4ce5-8bc3-84f54997b916-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zvz69\" (UID: \"e912041a-26ec-4ce5-8bc3-84f54997b916\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:06:29.782357 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.782333 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsdhr\" (UniqueName: \"kubernetes.io/projected/e912041a-26ec-4ce5-8bc3-84f54997b916-kube-api-access-nsdhr\") pod \"kuadrant-operator-controller-manager-86878c769f-zvz69\" (UID: \"e912041a-26ec-4ce5-8bc3-84f54997b916\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:06:29.867370 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:29.867256 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:06:30.016780 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:30.016755 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69"] Jul 10 13:06:30.019423 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:06:30.019398 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode912041a_26ec_4ce5_8bc3_84f54997b916.slice/crio-4b238d55efcdd92905d1890dccea2f8c87f15d4e3bc573060f4a6b3660929d16 WatchSource:0}: Error finding container 4b238d55efcdd92905d1890dccea2f8c87f15d4e3bc573060f4a6b3660929d16: Status 404 returned error can't find the container with id 4b238d55efcdd92905d1890dccea2f8c87f15d4e3bc573060f4a6b3660929d16 Jul 10 13:06:30.493570 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:30.493532 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" event={"ID":"e912041a-26ec-4ce5-8bc3-84f54997b916","Type":"ContainerStarted","Data":"3ad6d59d58374503137beb02540741c8655da195ae6a7d7e7afb2d917a470c2c"} Jul 10 13:06:30.493739 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:30.493589 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" event={"ID":"e912041a-26ec-4ce5-8bc3-84f54997b916","Type":"ContainerStarted","Data":"4b238d55efcdd92905d1890dccea2f8c87f15d4e3bc573060f4a6b3660929d16"} Jul 10 13:06:30.493739 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:30.493610 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:06:30.520816 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:30.520762 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" podStartSLOduration=1.520742874 podStartE2EDuration="1.520742874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:06:30.519373208 +0000 UTC m=+1241.405133824" watchObservedRunningTime="2026-07-10 13:06:30.520742874 +0000 UTC m=+1241.406503490" Jul 10 13:06:41.502174 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:06:41.502144 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zvz69" Jul 10 13:15:00.146397 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:00.146368 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f"] Jul 10 13:15:00.149926 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:00.149912 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" Jul 10 13:15:00.153508 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:00.153488 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sfczk\"" Jul 10 13:15:00.159514 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:00.159495 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" Jul 10 13:15:00.183157 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:00.183134 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f"] Jul 10 13:15:00.300842 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:00.300816 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f"] Jul 10 13:15:00.302828 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:15:00.302800 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61dad053_510c_4145_b221_f391f16f0549.slice/crio-4e623f893cd27fed6bd2be5bc6658a6e1c49105b872c0e954cb5fe99789c033e WatchSource:0}: Error finding container 4e623f893cd27fed6bd2be5bc6658a6e1c49105b872c0e954cb5fe99789c033e: Status 404 returned error can't find the container with id 4e623f893cd27fed6bd2be5bc6658a6e1c49105b872c0e954cb5fe99789c033e Jul 10 13:15:00.512706 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:00.512627 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" event={"ID":"61dad053-510c-4145-b221-f391f16f0549","Type":"ContainerStarted","Data":"4e623f893cd27fed6bd2be5bc6658a6e1c49105b872c0e954cb5fe99789c033e"} Jul 10 13:15:01.519565 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:01.519527 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" event={"ID":"61dad053-510c-4145-b221-f391f16f0549","Type":"ContainerStarted","Data":"033d37cf258d02a69c5622de160b1762f0a0b440077e3ff49664f27de4340b96"} Jul 10 13:15:01.538765 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:01.538712 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" podStartSLOduration=1.538696776 podStartE2EDuration="1.538696776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:15:01.53678644 +0000 UTC m=+1752.422547055" watchObservedRunningTime="2026-07-10 13:15:01.538696776 +0000 UTC m=+1752.424457391" Jul 10 13:15:15.346664 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.346630 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch"] Jul 10 13:15:15.352177 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.352151 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.356446 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.356424 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-72c69b0b-openshift-default-dockercfg-zfc46\"" Jul 10 13:15:15.398161 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.398133 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch"] Jul 10 13:15:15.428502 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.428470 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-credential-socket\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.428654 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.428515 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-socket\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.428654 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.428590 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-data\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.428654 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.428633 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-podinfo\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.428789 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.428658 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-certs\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.428789 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.428721 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skqt6\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-kube-api-access-skqt6\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.428789 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.428756 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istiod-ca-cert\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.428917 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.428804 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-token\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.428917 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.428836 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-envoy\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.529817 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.529774 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-data\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530004 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.529833 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-podinfo\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530004 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.529862 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-certs\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530004 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.529906 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skqt6\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-kube-api-access-skqt6\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530004 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.529935 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istiod-ca-cert\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530004 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.529989 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-token\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530363 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.530023 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-envoy\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530363 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.530064 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-credential-socket\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530363 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.530095 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-socket\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530363 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.530248 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-data\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530577 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.530524 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-certs\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530673 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.530649 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-credential-socket\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530734 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.530706 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istiod-ca-cert\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.530985 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.530965 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-socket\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.532442 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.532419 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-envoy\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.532541 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.532505 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-podinfo\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.540446 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.540420 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skqt6\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-kube-api-access-skqt6\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.541284 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.541264 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-token\") pod \"e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.664387 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.664357 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:15.809612 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.809583 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch"] Jul 10 13:15:15.810640 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:15:15.810606 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a91ab6a_0119_4c37_96f8_fbc9052d53cf.slice/crio-82eda387cc8200ac9659435f1a89f0067a041a09f796d462232a451963d5361f WatchSource:0}: Error finding container 82eda387cc8200ac9659435f1a89f0067a041a09f796d462232a451963d5361f: Status 404 returned error can't find the container with id 82eda387cc8200ac9659435f1a89f0067a041a09f796d462232a451963d5361f Jul 10 13:15:15.812856 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.812812 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:15:15.812929 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.812910 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:15:15.812972 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:15.812961 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:15:16.618313 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:16.618261 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" event={"ID":"7a91ab6a-0119-4c37-96f8-fbc9052d53cf","Type":"ContainerStarted","Data":"00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f"} Jul 10 13:15:16.618789 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:16.618350 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" event={"ID":"7a91ab6a-0119-4c37-96f8-fbc9052d53cf","Type":"ContainerStarted","Data":"82eda387cc8200ac9659435f1a89f0067a041a09f796d462232a451963d5361f"} Jul 10 13:15:16.653219 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:16.653172 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" podStartSLOduration=1.653155505 podStartE2EDuration="1.653155505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:15:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:15:16.648337944 +0000 UTC m=+1767.534098556" watchObservedRunningTime="2026-07-10 13:15:16.653155505 +0000 UTC m=+1767.538916146" Jul 10 13:15:16.664774 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:16.664746 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:16.669531 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:16.669512 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:17.626515 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:17.626488 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:17.627579 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:17.627555 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:21.777437 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:21.777402 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch"] Jul 10 13:15:21.777812 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:21.777613 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" podUID="7a91ab6a-0119-4c37-96f8-fbc9052d53cf" containerName="istio-proxy" containerID="cri-o://00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f" gracePeriod=30 Jul 10 13:15:28.529273 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.529249 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:28.662899 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.662871 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-envoy\") pod \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " Jul 10 13:15:28.663082 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.662905 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-credential-socket\") pod \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " Jul 10 13:15:28.663082 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.662950 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-socket\") pod \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " Jul 10 13:15:28.663082 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.662992 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-podinfo\") pod \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " Jul 10 13:15:28.663082 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663010 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-data\") pod \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " Jul 10 13:15:28.663082 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663039 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-certs\") pod \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " Jul 10 13:15:28.663082 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663065 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skqt6\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-kube-api-access-skqt6\") pod \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " Jul 10 13:15:28.663444 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663103 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-token\") pod \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " Jul 10 13:15:28.663444 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663131 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istiod-ca-cert\") pod \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\" (UID: \"7a91ab6a-0119-4c37-96f8-fbc9052d53cf\") " Jul 10 13:15:28.663444 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663181 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7a91ab6a-0119-4c37-96f8-fbc9052d53cf" (UID: "7a91ab6a-0119-4c37-96f8-fbc9052d53cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:28.663444 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663356 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7a91ab6a-0119-4c37-96f8-fbc9052d53cf" (UID: "7a91ab6a-0119-4c37-96f8-fbc9052d53cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:28.663444 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663371 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7a91ab6a-0119-4c37-96f8-fbc9052d53cf" (UID: "7a91ab6a-0119-4c37-96f8-fbc9052d53cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:28.663694 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663502 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-credential-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:28.663694 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663552 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:28.663694 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663570 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-workload-certs\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:28.663845 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.663750 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7a91ab6a-0119-4c37-96f8-fbc9052d53cf" (UID: "7a91ab6a-0119-4c37-96f8-fbc9052d53cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:15:28.664097 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.664075 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-data" (OuterVolumeSpecName: "istio-data") pod "7a91ab6a-0119-4c37-96f8-fbc9052d53cf" (UID: "7a91ab6a-0119-4c37-96f8-fbc9052d53cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:28.665231 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.665209 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7a91ab6a-0119-4c37-96f8-fbc9052d53cf" (UID: "7a91ab6a-0119-4c37-96f8-fbc9052d53cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:28.665377 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.665284 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-token" (OuterVolumeSpecName: "istio-token") pod "7a91ab6a-0119-4c37-96f8-fbc9052d53cf" (UID: "7a91ab6a-0119-4c37-96f8-fbc9052d53cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:15:28.665579 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.665561 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7a91ab6a-0119-4c37-96f8-fbc9052d53cf" (UID: "7a91ab6a-0119-4c37-96f8-fbc9052d53cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 13:15:28.665579 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.665570 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-kube-api-access-skqt6" (OuterVolumeSpecName: "kube-api-access-skqt6") pod "7a91ab6a-0119-4c37-96f8-fbc9052d53cf" (UID: "7a91ab6a-0119-4c37-96f8-fbc9052d53cf"). InnerVolumeSpecName "kube-api-access-skqt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:15:28.700157 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.700131 2571 generic.go:356] "Generic (PLEG): container finished" podID="7a91ab6a-0119-4c37-96f8-fbc9052d53cf" containerID="00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f" exitCode=0 Jul 10 13:15:28.700280 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.700183 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" Jul 10 13:15:28.700280 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.700215 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" event={"ID":"7a91ab6a-0119-4c37-96f8-fbc9052d53cf","Type":"ContainerDied","Data":"00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f"} Jul 10 13:15:28.700280 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.700266 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch" event={"ID":"7a91ab6a-0119-4c37-96f8-fbc9052d53cf","Type":"ContainerDied","Data":"82eda387cc8200ac9659435f1a89f0067a041a09f796d462232a451963d5361f"} Jul 10 13:15:28.700460 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.700301 2571 scope.go:117] "RemoveContainer" containerID="00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f" Jul 10 13:15:28.709886 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.709869 2571 scope.go:117] "RemoveContainer" containerID="00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f" Jul 10 13:15:28.710117 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:15:28.710100 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f\": container with ID starting with 00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f not found: ID does not exist" containerID="00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f" Jul 10 13:15:28.710167 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.710129 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f"} err="failed to get container status \"00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f\": rpc error: code = NotFound desc = could not find container \"00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f\": container with ID starting with 00c8e65d756b5c3904cc859a77d67160f07dc965ee1f2a825b4345f6d2b79e8f not found: ID does not exist" Jul 10 13:15:28.725949 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.725921 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch"] Jul 10 13:15:28.732779 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.732751 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-72c69b0b-openshift-default-77b59cf48f-xv8ch"] Jul 10 13:15:28.764507 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.764481 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-envoy\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:28.764507 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.764505 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-podinfo\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:28.764642 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.764518 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-data\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:28.764642 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.764531 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skqt6\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-kube-api-access-skqt6\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:28.764642 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.764544 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istio-token\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:28.764642 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:28.764556 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a91ab6a-0119-4c37-96f8-fbc9052d53cf-istiod-ca-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:29.538106 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:29.538073 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a91ab6a-0119-4c37-96f8-fbc9052d53cf" path="/var/lib/kubelet/pods/7a91ab6a-0119-4c37-96f8-fbc9052d53cf/volumes" Jul 10 13:15:46.613417 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.613139 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c"] Jul 10 13:15:46.613859 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.613839 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a91ab6a-0119-4c37-96f8-fbc9052d53cf" containerName="istio-proxy" Jul 10 13:15:46.613904 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.613863 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a91ab6a-0119-4c37-96f8-fbc9052d53cf" containerName="istio-proxy" Jul 10 13:15:46.613985 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.613973 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a91ab6a-0119-4c37-96f8-fbc9052d53cf" containerName="istio-proxy" Jul 10 13:15:46.618938 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.618919 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.623053 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.623030 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-68840b9c-openshift-default-dockercfg-tmm7p\"" Jul 10 13:15:46.636967 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.636942 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c"] Jul 10 13:15:46.730967 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.730935 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-istio-token\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.731126 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.730983 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d13486-b7f1-429a-a123-13fc31d33b87-istiod-ca-cert\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.731126 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.731015 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-data\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.731126 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.731053 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d13486-b7f1-429a-a123-13fc31d33b87-istio-podinfo\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.731126 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.731075 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-envoy\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.731126 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.731119 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-certs\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.731374 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.731205 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-692qj\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-kube-api-access-692qj\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.731374 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.731274 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-socket\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.731374 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.731350 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-credential-socket\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832256 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832227 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-socket\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832447 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832267 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-credential-socket\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832447 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832405 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-istio-token\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832579 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832486 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d13486-b7f1-429a-a123-13fc31d33b87-istiod-ca-cert\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832579 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832527 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-data\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832579 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832557 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d13486-b7f1-429a-a123-13fc31d33b87-istio-podinfo\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832717 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832675 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-credential-socket\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832717 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832680 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-envoy\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832830 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832739 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-socket\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832830 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832752 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-certs\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832830 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832819 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-692qj\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-kube-api-access-692qj\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.832974 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.832929 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-data\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.833182 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.833157 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-certs\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.833338 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.833228 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d13486-b7f1-429a-a123-13fc31d33b87-istiod-ca-cert\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.834929 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.834905 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-envoy\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.835134 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.835120 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d13486-b7f1-429a-a123-13fc31d33b87-istio-podinfo\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.840375 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.840354 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-istio-token\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.840650 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.840635 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-692qj\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-kube-api-access-692qj\") pod \"e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:46.929823 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:46.929801 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:47.079038 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:47.079008 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c"] Jul 10 13:15:47.080590 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:15:47.080552 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61d13486_b7f1_429a_a123_13fc31d33b87.slice/crio-d395f0dae4eef30847fd4d42c0304f58195e37e606c3feeb9083f402e24a3c00 WatchSource:0}: Error finding container d395f0dae4eef30847fd4d42c0304f58195e37e606c3feeb9083f402e24a3c00: Status 404 returned error can't find the container with id d395f0dae4eef30847fd4d42c0304f58195e37e606c3feeb9083f402e24a3c00 Jul 10 13:15:47.083150 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:47.083118 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:15:47.083257 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:47.083194 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:15:47.083257 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:47.083238 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:15:47.812664 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:47.812625 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" event={"ID":"61d13486-b7f1-429a-a123-13fc31d33b87","Type":"ContainerStarted","Data":"9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8"} Jul 10 13:15:47.813057 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:47.812684 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" event={"ID":"61d13486-b7f1-429a-a123-13fc31d33b87","Type":"ContainerStarted","Data":"d395f0dae4eef30847fd4d42c0304f58195e37e606c3feeb9083f402e24a3c00"} Jul 10 13:15:47.849446 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:47.849388 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" podStartSLOduration=1.8493695510000001 podStartE2EDuration="1.849369551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:15:47.848126961 +0000 UTC m=+1798.733887576" watchObservedRunningTime="2026-07-10 13:15:47.849369551 +0000 UTC m=+1798.735130178" Jul 10 13:15:47.930436 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:47.930408 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:48.935620 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:48.935591 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:49.826076 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:49.826041 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:49.826877 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:49.826858 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:52.372982 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:52.372943 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c"] Jul 10 13:15:52.850637 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:52.850569 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" podUID="61d13486-b7f1-429a-a123-13fc31d33b87" containerName="istio-proxy" containerID="cri-o://9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8" gracePeriod=30 Jul 10 13:15:58.824048 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.824025 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:58.885949 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.885916 2571 generic.go:356] "Generic (PLEG): container finished" podID="61d13486-b7f1-429a-a123-13fc31d33b87" containerID="9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8" exitCode=0 Jul 10 13:15:58.886103 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.885971 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" Jul 10 13:15:58.886103 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.886008 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" event={"ID":"61d13486-b7f1-429a-a123-13fc31d33b87","Type":"ContainerDied","Data":"9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8"} Jul 10 13:15:58.886103 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.886078 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c" event={"ID":"61d13486-b7f1-429a-a123-13fc31d33b87","Type":"ContainerDied","Data":"d395f0dae4eef30847fd4d42c0304f58195e37e606c3feeb9083f402e24a3c00"} Jul 10 13:15:58.886103 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.886101 2571 scope.go:117] "RemoveContainer" containerID="9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8" Jul 10 13:15:58.895550 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.895534 2571 scope.go:117] "RemoveContainer" containerID="9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8" Jul 10 13:15:58.895807 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:15:58.895791 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8\": container with ID starting with 9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8 not found: ID does not exist" containerID="9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8" Jul 10 13:15:58.895852 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.895815 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8"} err="failed to get container status \"9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8\": rpc error: code = NotFound desc = could not find container \"9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8\": container with ID starting with 9e670d51858590c3c550a6738787fa33e50afd7a1e2633a6b6d00319860b4cf8 not found: ID does not exist" Jul 10 13:15:58.943938 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.943916 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-socket\") pod \"61d13486-b7f1-429a-a123-13fc31d33b87\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " Jul 10 13:15:58.944051 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.943952 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d13486-b7f1-429a-a123-13fc31d33b87-istio-podinfo\") pod \"61d13486-b7f1-429a-a123-13fc31d33b87\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " Jul 10 13:15:58.944051 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944009 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-envoy\") pod \"61d13486-b7f1-429a-a123-13fc31d33b87\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " Jul 10 13:15:58.944051 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944043 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-certs\") pod \"61d13486-b7f1-429a-a123-13fc31d33b87\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " Jul 10 13:15:58.944220 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944068 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-data\") pod \"61d13486-b7f1-429a-a123-13fc31d33b87\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " Jul 10 13:15:58.944220 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944151 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-credential-socket\") pod \"61d13486-b7f1-429a-a123-13fc31d33b87\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " Jul 10 13:15:58.944220 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944201 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-istio-token\") pod \"61d13486-b7f1-429a-a123-13fc31d33b87\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " Jul 10 13:15:58.944409 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944262 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-692qj\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-kube-api-access-692qj\") pod \"61d13486-b7f1-429a-a123-13fc31d33b87\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " Jul 10 13:15:58.944409 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944313 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d13486-b7f1-429a-a123-13fc31d33b87-istiod-ca-cert\") pod \"61d13486-b7f1-429a-a123-13fc31d33b87\" (UID: \"61d13486-b7f1-429a-a123-13fc31d33b87\") " Jul 10 13:15:58.944409 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944196 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61d13486-b7f1-429a-a123-13fc31d33b87" (UID: "61d13486-b7f1-429a-a123-13fc31d33b87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:58.944563 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944399 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61d13486-b7f1-429a-a123-13fc31d33b87" (UID: "61d13486-b7f1-429a-a123-13fc31d33b87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:58.944563 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944472 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61d13486-b7f1-429a-a123-13fc31d33b87" (UID: "61d13486-b7f1-429a-a123-13fc31d33b87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:58.944686 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944660 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-credential-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:58.944731 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944697 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:58.944731 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944713 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-workload-certs\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:58.944731 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944672 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61d13486-b7f1-429a-a123-13fc31d33b87-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61d13486-b7f1-429a-a123-13fc31d33b87" (UID: "61d13486-b7f1-429a-a123-13fc31d33b87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:15:58.945008 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.944981 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-data" (OuterVolumeSpecName: "istio-data") pod "61d13486-b7f1-429a-a123-13fc31d33b87" (UID: "61d13486-b7f1-429a-a123-13fc31d33b87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:58.946395 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.946371 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-istio-token" (OuterVolumeSpecName: "istio-token") pod "61d13486-b7f1-429a-a123-13fc31d33b87" (UID: "61d13486-b7f1-429a-a123-13fc31d33b87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:15:58.946543 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.946523 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61d13486-b7f1-429a-a123-13fc31d33b87-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61d13486-b7f1-429a-a123-13fc31d33b87" (UID: "61d13486-b7f1-429a-a123-13fc31d33b87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 13:15:58.946780 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.946752 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-kube-api-access-692qj" (OuterVolumeSpecName: "kube-api-access-692qj") pod "61d13486-b7f1-429a-a123-13fc31d33b87" (UID: "61d13486-b7f1-429a-a123-13fc31d33b87"). InnerVolumeSpecName "kube-api-access-692qj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:15:58.946865 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:58.946809 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61d13486-b7f1-429a-a123-13fc31d33b87" (UID: "61d13486-b7f1-429a-a123-13fc31d33b87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:15:59.045464 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:59.045430 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-data\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:59.045464 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:59.045461 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-istio-token\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:59.045464 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:59.045471 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-692qj\" (UniqueName: \"kubernetes.io/projected/61d13486-b7f1-429a-a123-13fc31d33b87-kube-api-access-692qj\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:59.045638 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:59.045479 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d13486-b7f1-429a-a123-13fc31d33b87-istiod-ca-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:59.045638 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:59.045488 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d13486-b7f1-429a-a123-13fc31d33b87-istio-podinfo\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:59.045638 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:59.045496 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d13486-b7f1-429a-a123-13fc31d33b87-istio-envoy\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:15:59.213866 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:59.213839 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c"] Jul 10 13:15:59.216396 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:59.216378 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-68840b9c-openshift-default-6fbff97799-x2t6c"] Jul 10 13:15:59.538212 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:15:59.538133 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61d13486-b7f1-429a-a123-13fc31d33b87" path="/var/lib/kubelet/pods/61d13486-b7f1-429a-a123-13fc31d33b87/volumes" Jul 10 13:16:00.013435 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:00.013404 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f"] Jul 10 13:16:00.013833 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:00.013602 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" podUID="61dad053-510c-4145-b221-f391f16f0549" containerName="cleanup" containerID="cri-o://033d37cf258d02a69c5622de160b1762f0a0b440077e3ff49664f27de4340b96" gracePeriod=30 Jul 10 13:16:30.070730 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:30.070700 2571 generic.go:356] "Generic (PLEG): container finished" podID="61dad053-510c-4145-b221-f391f16f0549" containerID="033d37cf258d02a69c5622de160b1762f0a0b440077e3ff49664f27de4340b96" exitCode=137 Jul 10 13:16:30.071063 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:30.070770 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" event={"ID":"61dad053-510c-4145-b221-f391f16f0549","Type":"ContainerDied","Data":"033d37cf258d02a69c5622de160b1762f0a0b440077e3ff49664f27de4340b96"} Jul 10 13:16:30.206429 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:30.206407 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" Jul 10 13:16:31.077922 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:31.077895 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" Jul 10 13:16:31.077922 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:31.077903 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f" event={"ID":"61dad053-510c-4145-b221-f391f16f0549","Type":"ContainerDied","Data":"4e623f893cd27fed6bd2be5bc6658a6e1c49105b872c0e954cb5fe99789c033e"} Jul 10 13:16:31.078442 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:31.077967 2571 scope.go:117] "RemoveContainer" containerID="033d37cf258d02a69c5622de160b1762f0a0b440077e3ff49664f27de4340b96" Jul 10 13:16:31.100805 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:31.100777 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f"] Jul 10 13:16:31.104736 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:31.104707 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728155-g477f"] Jul 10 13:16:31.535863 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:16:31.535834 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61dad053-510c-4145-b221-f391f16f0549" path="/var/lib/kubelet/pods/61dad053-510c-4145-b221-f391f16f0549/volumes" Jul 10 13:17:40.417450 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.417363 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm"] Jul 10 13:17:40.418053 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.418033 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61d13486-b7f1-429a-a123-13fc31d33b87" containerName="istio-proxy" Jul 10 13:17:40.418136 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.418056 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="61d13486-b7f1-429a-a123-13fc31d33b87" containerName="istio-proxy" Jul 10 13:17:40.418136 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.418073 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61dad053-510c-4145-b221-f391f16f0549" containerName="cleanup" Jul 10 13:17:40.418136 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.418081 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="61dad053-510c-4145-b221-f391f16f0549" containerName="cleanup" Jul 10 13:17:40.418330 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.418218 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="61d13486-b7f1-429a-a123-13fc31d33b87" containerName="istio-proxy" Jul 10 13:17:40.418330 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.418233 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="61dad053-510c-4145-b221-f391f16f0549" containerName="cleanup" Jul 10 13:17:40.421522 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.421501 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.427832 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.427805 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-n5dl8\"" Jul 10 13:17:40.428848 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.428668 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fb9fe4e8-serving-cert\"" Jul 10 13:17:40.428848 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.428712 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 13:17:40.428848 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.428824 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 13:17:40.440382 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.440348 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm"] Jul 10 13:17:40.508351 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.508320 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b11c3045-978c-4c6e-aa36-158464246c47-openshift-service-ca\") pod \"maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.508519 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.508361 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b11c3045-978c-4c6e-aa36-158464246c47-maas-api-tls\") pod \"maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.508519 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.508452 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfqkc\" (UniqueName: \"kubernetes.io/projected/b11c3045-978c-4c6e-aa36-158464246c47-kube-api-access-cfqkc\") pod \"maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.609222 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.609189 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfqkc\" (UniqueName: \"kubernetes.io/projected/b11c3045-978c-4c6e-aa36-158464246c47-kube-api-access-cfqkc\") pod \"maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.609494 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.609473 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b11c3045-978c-4c6e-aa36-158464246c47-openshift-service-ca\") pod \"maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.609550 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.609539 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b11c3045-978c-4c6e-aa36-158464246c47-maas-api-tls\") pod \"maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.610079 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.610062 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b11c3045-978c-4c6e-aa36-158464246c47-openshift-service-ca\") pod \"maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.611899 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.611878 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b11c3045-978c-4c6e-aa36-158464246c47-maas-api-tls\") pod \"maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.617687 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.617664 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfqkc\" (UniqueName: \"kubernetes.io/projected/b11c3045-978c-4c6e-aa36-158464246c47-kube-api-access-cfqkc\") pod \"maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.732681 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.732596 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:40.892885 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:17:40.892852 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11c3045_978c_4c6e_aa36_158464246c47.slice/crio-3bec81aeebfb961b8718ab5c79b626c6b96fab2e97552734ce6be744a21eeb4a WatchSource:0}: Error finding container 3bec81aeebfb961b8718ab5c79b626c6b96fab2e97552734ce6be744a21eeb4a: Status 404 returned error can't find the container with id 3bec81aeebfb961b8718ab5c79b626c6b96fab2e97552734ce6be744a21eeb4a Jul 10 13:17:40.894232 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.894215 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 13:17:40.905649 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:40.905621 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm"] Jul 10 13:17:41.496603 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:41.496566 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" event={"ID":"b11c3045-978c-4c6e-aa36-158464246c47","Type":"ContainerStarted","Data":"3bec81aeebfb961b8718ab5c79b626c6b96fab2e97552734ce6be744a21eeb4a"} Jul 10 13:17:43.510368 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:43.510326 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" event={"ID":"b11c3045-978c-4c6e-aa36-158464246c47","Type":"ContainerStarted","Data":"6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931"} Jul 10 13:17:43.510850 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:43.510384 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:43.536868 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:43.536820 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" podStartSLOduration=1.550369289 podStartE2EDuration="3.536805397s" totalImagesPullingTime="1.986436108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:17:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:17:43.534393359 +0000 UTC m=+1914.420153975" watchObservedRunningTime="2026-07-10 13:17:43.536805397 +0000 UTC m=+1914.422566011" Jul 10 13:17:49.522640 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:49.522603 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:52.974655 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:52.974600 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm"] Jul 10 13:17:52.975127 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:52.974849 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" podUID="b11c3045-978c-4c6e-aa36-158464246c47" containerName="maas-api" containerID="cri-o://6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931" gracePeriod=30 Jul 10 13:17:53.230450 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.230381 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:53.335968 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.335934 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b11c3045-978c-4c6e-aa36-158464246c47-openshift-service-ca\") pod \"b11c3045-978c-4c6e-aa36-158464246c47\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " Jul 10 13:17:53.336165 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.335978 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b11c3045-978c-4c6e-aa36-158464246c47-maas-api-tls\") pod \"b11c3045-978c-4c6e-aa36-158464246c47\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " Jul 10 13:17:53.336165 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.336053 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfqkc\" (UniqueName: \"kubernetes.io/projected/b11c3045-978c-4c6e-aa36-158464246c47-kube-api-access-cfqkc\") pod \"b11c3045-978c-4c6e-aa36-158464246c47\" (UID: \"b11c3045-978c-4c6e-aa36-158464246c47\") " Jul 10 13:17:53.336394 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.336366 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b11c3045-978c-4c6e-aa36-158464246c47-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b11c3045-978c-4c6e-aa36-158464246c47" (UID: "b11c3045-978c-4c6e-aa36-158464246c47"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:17:53.338273 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.338244 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b11c3045-978c-4c6e-aa36-158464246c47-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b11c3045-978c-4c6e-aa36-158464246c47" (UID: "b11c3045-978c-4c6e-aa36-158464246c47"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 13:17:53.338401 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.338333 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b11c3045-978c-4c6e-aa36-158464246c47-kube-api-access-cfqkc" (OuterVolumeSpecName: "kube-api-access-cfqkc") pod "b11c3045-978c-4c6e-aa36-158464246c47" (UID: "b11c3045-978c-4c6e-aa36-158464246c47"). InnerVolumeSpecName "kube-api-access-cfqkc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:17:53.437592 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.437549 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b11c3045-978c-4c6e-aa36-158464246c47-openshift-service-ca\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:17:53.437592 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.437586 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b11c3045-978c-4c6e-aa36-158464246c47-maas-api-tls\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:17:53.437797 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.437600 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfqkc\" (UniqueName: \"kubernetes.io/projected/b11c3045-978c-4c6e-aa36-158464246c47-kube-api-access-cfqkc\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:17:53.576989 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.576950 2571 generic.go:356] "Generic (PLEG): container finished" podID="b11c3045-978c-4c6e-aa36-158464246c47" containerID="6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931" exitCode=0 Jul 10 13:17:53.577163 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.577011 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" Jul 10 13:17:53.577163 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.577029 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" event={"ID":"b11c3045-978c-4c6e-aa36-158464246c47","Type":"ContainerDied","Data":"6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931"} Jul 10 13:17:53.577163 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.577106 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm" event={"ID":"b11c3045-978c-4c6e-aa36-158464246c47","Type":"ContainerDied","Data":"3bec81aeebfb961b8718ab5c79b626c6b96fab2e97552734ce6be744a21eeb4a"} Jul 10 13:17:53.577163 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.577134 2571 scope.go:117] "RemoveContainer" containerID="6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931" Jul 10 13:17:53.586912 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.586893 2571 scope.go:117] "RemoveContainer" containerID="6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931" Jul 10 13:17:53.587200 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:17:53.587181 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931\": container with ID starting with 6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931 not found: ID does not exist" containerID="6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931" Jul 10 13:17:53.587246 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.587207 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931"} err="failed to get container status \"6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931\": rpc error: code = NotFound desc = could not find container \"6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931\": container with ID starting with 6a2a8647b1512be7b17219c53ea4fd923d24eaf8bb0e96dbc739d7d840fad931 not found: ID does not exist" Jul 10 13:17:53.610448 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.610426 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm"] Jul 10 13:17:53.615654 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:53.615631 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fb9fe4e8-78d74948bb-8lwgm"] Jul 10 13:17:55.536435 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:17:55.536400 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b11c3045-978c-4c6e-aa36-158464246c47" path="/var/lib/kubelet/pods/b11c3045-978c-4c6e-aa36-158464246c47/volumes" Jul 10 13:21:11.635871 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.635833 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j"] Jul 10 13:21:11.638611 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.636329 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b11c3045-978c-4c6e-aa36-158464246c47" containerName="maas-api" Jul 10 13:21:11.638611 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.636345 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b11c3045-978c-4c6e-aa36-158464246c47" containerName="maas-api" Jul 10 13:21:11.638611 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.636433 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b11c3045-978c-4c6e-aa36-158464246c47" containerName="maas-api" Jul 10 13:21:11.639521 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.639499 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.642502 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.642481 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-76c01a-openshift-default-dockercfg-f64pk\"" Jul 10 13:21:11.651839 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.651817 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j"] Jul 10 13:21:11.778696 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.778655 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-credential-socket\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.778905 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.778721 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-socket\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.778905 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.778816 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snfhr\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-kube-api-access-snfhr\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.778905 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.778898 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-podinfo\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.779075 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.778945 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d271dba-de20-4d38-a6ed-af0f894c9c25-istiod-ca-cert\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.779075 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.778982 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-data\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.779075 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.779009 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-token\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.779183 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.779081 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-envoy\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.779183 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.779110 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-certs\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.879943 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.879908 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snfhr\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-kube-api-access-snfhr\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880105 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.879963 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-podinfo\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880105 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.879992 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d271dba-de20-4d38-a6ed-af0f894c9c25-istiod-ca-cert\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880198 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880126 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-data\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880198 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-token\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880198 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880191 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-envoy\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880387 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880224 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-certs\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880448 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880370 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-credential-socket\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880577 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880551 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-socket\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880722 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880588 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-data\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880722 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880645 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-certs\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880846 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880763 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d271dba-de20-4d38-a6ed-af0f894c9c25-istiod-ca-cert\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880932 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880910 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-socket\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.880976 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.880919 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-credential-socket\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.882473 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.882453 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-envoy\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.882689 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.882662 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-podinfo\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.887857 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.887802 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-token\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.887966 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.887855 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snfhr\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-kube-api-access-snfhr\") pod \"e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:11.951335 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:11.951312 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:12.103157 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:21:12.103123 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d271dba_de20_4d38_a6ed_af0f894c9c25.slice/crio-f48acf602fd836cb3473d0bd17b8ae5574a50afeceb53f11e3d81a9a44717199 WatchSource:0}: Error finding container f48acf602fd836cb3473d0bd17b8ae5574a50afeceb53f11e3d81a9a44717199: Status 404 returned error can't find the container with id f48acf602fd836cb3473d0bd17b8ae5574a50afeceb53f11e3d81a9a44717199 Jul 10 13:21:12.105558 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:12.105514 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:21:12.105676 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:12.105596 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:21:12.105676 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:12.105625 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:21:12.116360 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:12.116337 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j"] Jul 10 13:21:12.763135 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:12.763101 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" event={"ID":"6d271dba-de20-4d38-a6ed-af0f894c9c25","Type":"ContainerStarted","Data":"ebbf33879e6fa3e87de28efdeb1b2eaef7c53899f68ff39b9eb044a988a30847"} Jul 10 13:21:12.763523 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:12.763155 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" event={"ID":"6d271dba-de20-4d38-a6ed-af0f894c9c25","Type":"ContainerStarted","Data":"f48acf602fd836cb3473d0bd17b8ae5574a50afeceb53f11e3d81a9a44717199"} Jul 10 13:21:12.786075 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:12.786024 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" podStartSLOduration=1.786009223 podStartE2EDuration="1.786009223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:21:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:21:12.784393051 +0000 UTC m=+2123.670153666" watchObservedRunningTime="2026-07-10 13:21:12.786009223 +0000 UTC m=+2123.671769837" Jul 10 13:21:12.951915 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:12.951884 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:13.957238 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:13.957208 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:14.774507 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:14.774472 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:14.775401 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:14.775384 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:29.776470 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:29.776429 2571 patch_prober.go:28] interesting pod/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 10 13:21:29.776897 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:29.776492 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 10 13:21:29.871432 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:29.871358 2571 generic.go:356] "Generic (PLEG): container finished" podID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerID="ebbf33879e6fa3e87de28efdeb1b2eaef7c53899f68ff39b9eb044a988a30847" exitCode=0 Jul 10 13:21:29.871432 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:29.871420 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" event={"ID":"6d271dba-de20-4d38-a6ed-af0f894c9c25","Type":"ContainerDied","Data":"ebbf33879e6fa3e87de28efdeb1b2eaef7c53899f68ff39b9eb044a988a30847"} Jul 10 13:21:29.871886 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:29.871870 2571 scope.go:117] "RemoveContainer" containerID="ebbf33879e6fa3e87de28efdeb1b2eaef7c53899f68ff39b9eb044a988a30847" Jul 10 13:21:29.872889 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:29.872857 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:21:29.872946 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:29.872930 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:21:29.872983 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:29.872960 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:21:30.878458 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:30.878423 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" event={"ID":"6d271dba-de20-4d38-a6ed-af0f894c9c25","Type":"ContainerStarted","Data":"d67473539b1e76e9c9ea0246be87e906c032273a2ef433b213dfb7de36282adb"} Jul 10 13:21:30.951691 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:30.951654 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:30.951957 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:30.951923 2571 patch_prober.go:28] interesting pod/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 10 13:21:30.952044 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:30.951987 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 10 13:21:31.952659 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:31.952621 2571 patch_prober.go:28] interesting pod/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 10 13:21:31.953046 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:31.952689 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 10 13:21:32.952244 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:32.952210 2571 patch_prober.go:28] interesting pod/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 10 13:21:32.952435 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:32.952266 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 10 13:21:33.952771 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:33.952726 2571 patch_prober.go:28] interesting pod/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 10 13:21:33.953258 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:33.952795 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 10 13:21:34.951793 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:34.951754 2571 patch_prober.go:28] interesting pod/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 10 13:21:34.951968 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:34.951830 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 10 13:21:35.955849 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:35.955812 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:35.956373 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:35.956166 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:21:35.956944 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:21:35.956925 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:22:07.443838 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.443801 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2"] Jul 10 13:22:07.447580 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.447563 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.450890 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.450870 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-reuse-0fbd6d-openshift-default-dockercfg-brv5w\"" Jul 10 13:22:07.455001 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.454967 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-socket\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.455120 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.455009 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-token\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.455120 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.455052 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-certs\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.455120 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.455088 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-envoy\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.455278 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.455124 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhmt7\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-kube-api-access-zhmt7\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.455278 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.455183 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-data\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.455278 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.455226 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-credential-socket\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.455479 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.455327 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e52891-24cd-4aeb-8031-3d3af15ad26c-istiod-ca-cert\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.455479 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.455354 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-podinfo\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.458072 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.458051 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2"] Jul 10 13:22:07.556749 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.556680 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhmt7\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-kube-api-access-zhmt7\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.557002 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.556928 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-data\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.557108 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557050 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-credential-socket\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.557174 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557112 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e52891-24cd-4aeb-8031-3d3af15ad26c-istiod-ca-cert\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.557174 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557163 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-podinfo\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.557665 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557355 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-socket\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.557665 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557386 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-token\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.557665 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557410 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-data\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.557665 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557432 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-certs\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.557949 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557881 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-envoy\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.558016 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557948 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-certs\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.558016 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557954 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-socket\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.558016 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.557980 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e52891-24cd-4aeb-8031-3d3af15ad26c-istiod-ca-cert\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.558236 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.558212 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-credential-socket\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.559855 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.559832 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-podinfo\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.559954 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.559941 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-envoy\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.569254 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.569234 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-token\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.569517 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.569501 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhmt7\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-kube-api-access-zhmt7\") pod \"e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.758492 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.758404 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:07.910322 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.910008 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2"] Jul 10 13:22:07.912478 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.912430 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:22:07.912598 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.912576 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:22:07.912798 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:07.912754 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:22:08.102514 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:08.102431 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" event={"ID":"50e52891-24cd-4aeb-8031-3d3af15ad26c","Type":"ContainerStarted","Data":"9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092"} Jul 10 13:22:08.102514 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:08.102489 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" event={"ID":"50e52891-24cd-4aeb-8031-3d3af15ad26c","Type":"ContainerStarted","Data":"7ffb87ed55e0d5a19f40a2ac4f05fe1acc2811b242bcbe1930872eef89515c9a"} Jul 10 13:22:08.122718 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:08.122658 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" podStartSLOduration=1.122637508 podStartE2EDuration="1.122637508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:22:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:22:08.120972281 +0000 UTC m=+2179.006732894" watchObservedRunningTime="2026-07-10 13:22:08.122637508 +0000 UTC m=+2179.008398126" Jul 10 13:22:08.759135 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:08.759104 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:08.763706 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:08.763683 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:09.109057 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:09.108975 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:09.109765 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:09.109745 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:31.245251 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:31.245170 2571 generic.go:356] "Generic (PLEG): container finished" podID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerID="9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092" exitCode=0 Jul 10 13:22:31.245757 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:31.245248 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" event={"ID":"50e52891-24cd-4aeb-8031-3d3af15ad26c","Type":"ContainerDied","Data":"9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092"} Jul 10 13:22:31.245824 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:31.245802 2571 scope.go:117] "RemoveContainer" containerID="9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092" Jul 10 13:22:31.246924 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:31.246893 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:22:31.247031 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:31.246983 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:22:31.247099 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:31.247031 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:22:32.254898 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:32.254857 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" event={"ID":"50e52891-24cd-4aeb-8031-3d3af15ad26c","Type":"ContainerStarted","Data":"9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955"} Jul 10 13:22:32.759099 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:32.759060 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:32.759350 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:32.759321 2571 patch_prober.go:28] interesting pod/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 10 13:22:32.759438 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:32.759388 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 10 13:22:33.759309 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:33.759261 2571 patch_prober.go:28] interesting pod/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 10 13:22:33.759718 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:33.759349 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 10 13:22:34.759619 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:34.759574 2571 patch_prober.go:28] interesting pod/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 10 13:22:34.760102 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:34.759639 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 10 13:22:35.759575 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:35.759538 2571 patch_prober.go:28] interesting pod/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 10 13:22:35.759757 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:35.759602 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 10 13:22:36.763034 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:36.763003 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:36.763428 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:36.763379 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:22:36.763964 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:22:36.763943 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:23:34.731493 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:34.731461 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2"] Jul 10 13:23:34.732031 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:34.731737 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" containerID="cri-o://9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955" gracePeriod=30 Jul 10 13:23:36.766204 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:36.766170 2571 patch_prober.go:28] interesting pod/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 13:23:36.766604 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:36.766233 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 13:23:39.987454 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:39.987430 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:23:40.071854 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.071782 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-data\") pod \"50e52891-24cd-4aeb-8031-3d3af15ad26c\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " Jul 10 13:23:40.071854 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.071849 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-podinfo\") pod \"50e52891-24cd-4aeb-8031-3d3af15ad26c\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " Jul 10 13:23:40.072068 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.071893 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-socket\") pod \"50e52891-24cd-4aeb-8031-3d3af15ad26c\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " Jul 10 13:23:40.072068 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.071933 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhmt7\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-kube-api-access-zhmt7\") pod \"50e52891-24cd-4aeb-8031-3d3af15ad26c\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " Jul 10 13:23:40.072068 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.071951 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e52891-24cd-4aeb-8031-3d3af15ad26c-istiod-ca-cert\") pod \"50e52891-24cd-4aeb-8031-3d3af15ad26c\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " Jul 10 13:23:40.072068 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.071974 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-envoy\") pod \"50e52891-24cd-4aeb-8031-3d3af15ad26c\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " Jul 10 13:23:40.072068 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.071996 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-token\") pod \"50e52891-24cd-4aeb-8031-3d3af15ad26c\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " Jul 10 13:23:40.072068 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072020 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-certs\") pod \"50e52891-24cd-4aeb-8031-3d3af15ad26c\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " Jul 10 13:23:40.072068 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072065 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-credential-socket\") pod \"50e52891-24cd-4aeb-8031-3d3af15ad26c\" (UID: \"50e52891-24cd-4aeb-8031-3d3af15ad26c\") " Jul 10 13:23:40.072448 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072139 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "50e52891-24cd-4aeb-8031-3d3af15ad26c" (UID: "50e52891-24cd-4aeb-8031-3d3af15ad26c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:23:40.072448 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072369 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "50e52891-24cd-4aeb-8031-3d3af15ad26c" (UID: "50e52891-24cd-4aeb-8031-3d3af15ad26c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:23:40.072448 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072419 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50e52891-24cd-4aeb-8031-3d3af15ad26c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "50e52891-24cd-4aeb-8031-3d3af15ad26c" (UID: "50e52891-24cd-4aeb-8031-3d3af15ad26c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:23:40.072613 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072555 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:23:40.072613 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072576 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e52891-24cd-4aeb-8031-3d3af15ad26c-istiod-ca-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:23:40.072613 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072586 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "50e52891-24cd-4aeb-8031-3d3af15ad26c" (UID: "50e52891-24cd-4aeb-8031-3d3af15ad26c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:23:40.072613 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072590 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-credential-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:23:40.072804 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.072740 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-data" (OuterVolumeSpecName: "istio-data") pod "50e52891-24cd-4aeb-8031-3d3af15ad26c" (UID: "50e52891-24cd-4aeb-8031-3d3af15ad26c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:23:40.074071 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.074045 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "50e52891-24cd-4aeb-8031-3d3af15ad26c" (UID: "50e52891-24cd-4aeb-8031-3d3af15ad26c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 13:23:40.074337 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.074312 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-kube-api-access-zhmt7" (OuterVolumeSpecName: "kube-api-access-zhmt7") pod "50e52891-24cd-4aeb-8031-3d3af15ad26c" (UID: "50e52891-24cd-4aeb-8031-3d3af15ad26c"). InnerVolumeSpecName "kube-api-access-zhmt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:23:40.074528 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.074509 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-token" (OuterVolumeSpecName: "istio-token") pod "50e52891-24cd-4aeb-8031-3d3af15ad26c" (UID: "50e52891-24cd-4aeb-8031-3d3af15ad26c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:23:40.074604 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.074581 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "50e52891-24cd-4aeb-8031-3d3af15ad26c" (UID: "50e52891-24cd-4aeb-8031-3d3af15ad26c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:23:40.173854 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.173817 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhmt7\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-kube-api-access-zhmt7\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:23:40.173854 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.173849 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-envoy\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:23:40.173854 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.173858 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-token\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:23:40.174075 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.173866 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-workload-certs\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:23:40.174075 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.173874 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-data\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:23:40.174075 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.173881 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e52891-24cd-4aeb-8031-3d3af15ad26c-istio-podinfo\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:23:40.663183 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.663147 2571 generic.go:356] "Generic (PLEG): container finished" podID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerID="9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955" exitCode=0 Jul 10 13:23:40.663364 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.663203 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" Jul 10 13:23:40.663364 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.663231 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" event={"ID":"50e52891-24cd-4aeb-8031-3d3af15ad26c","Type":"ContainerDied","Data":"9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955"} Jul 10 13:23:40.663364 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.663304 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2" event={"ID":"50e52891-24cd-4aeb-8031-3d3af15ad26c","Type":"ContainerDied","Data":"7ffb87ed55e0d5a19f40a2ac4f05fe1acc2811b242bcbe1930872eef89515c9a"} Jul 10 13:23:40.663364 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.663329 2571 scope.go:117] "RemoveContainer" containerID="9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955" Jul 10 13:23:40.673651 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.673634 2571 scope.go:117] "RemoveContainer" containerID="9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092" Jul 10 13:23:40.685273 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.685254 2571 scope.go:117] "RemoveContainer" containerID="9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955" Jul 10 13:23:40.685581 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:23:40.685561 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955\": container with ID starting with 9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955 not found: ID does not exist" containerID="9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955" Jul 10 13:23:40.685638 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.685591 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955"} err="failed to get container status \"9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955\": rpc error: code = NotFound desc = could not find container \"9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955\": container with ID starting with 9b663aab129999069ac954652ae7dd4d5679fac1449dbdfc34b3d57cbbb8f955 not found: ID does not exist" Jul 10 13:23:40.685638 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.685608 2571 scope.go:117] "RemoveContainer" containerID="9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092" Jul 10 13:23:40.685859 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:23:40.685841 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092\": container with ID starting with 9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092 not found: ID does not exist" containerID="9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092" Jul 10 13:23:40.685904 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.685866 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092"} err="failed to get container status \"9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092\": rpc error: code = NotFound desc = could not find container \"9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092\": container with ID starting with 9bb70b116b02e9c0aefad0625a93851cce7fbc9a847190a3df46761fc22ab092 not found: ID does not exist" Jul 10 13:23:40.687053 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.687036 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2"] Jul 10 13:23:40.692447 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:40.692426 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-reuse-0fbd6d-openshift-default-68878c4d59-pw2d2"] Jul 10 13:23:41.537444 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:23:41.537416 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" path="/var/lib/kubelet/pods/50e52891-24cd-4aeb-8031-3d3af15ad26c/volumes" Jul 10 13:26:26.530887 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:26.530856 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j"] Jul 10 13:26:26.531443 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:26.531121 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" containerID="cri-o://d67473539b1e76e9c9ea0246be87e906c032273a2ef433b213dfb7de36282adb" gracePeriod=30 Jul 10 13:26:31.681968 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.681935 2571 generic.go:356] "Generic (PLEG): container finished" podID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerID="d67473539b1e76e9c9ea0246be87e906c032273a2ef433b213dfb7de36282adb" exitCode=0 Jul 10 13:26:31.682463 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.682019 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" event={"ID":"6d271dba-de20-4d38-a6ed-af0f894c9c25","Type":"ContainerDied","Data":"d67473539b1e76e9c9ea0246be87e906c032273a2ef433b213dfb7de36282adb"} Jul 10 13:26:31.682463 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.682101 2571 scope.go:117] "RemoveContainer" containerID="ebbf33879e6fa3e87de28efdeb1b2eaef7c53899f68ff39b9eb044a988a30847" Jul 10 13:26:31.800782 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.800717 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:26:31.945003 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.944974 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-token\") pod \"6d271dba-de20-4d38-a6ed-af0f894c9c25\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " Jul 10 13:26:31.945003 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945008 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-podinfo\") pod \"6d271dba-de20-4d38-a6ed-af0f894c9c25\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " Jul 10 13:26:31.945198 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945030 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snfhr\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-kube-api-access-snfhr\") pod \"6d271dba-de20-4d38-a6ed-af0f894c9c25\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " Jul 10 13:26:31.945198 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945172 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-certs\") pod \"6d271dba-de20-4d38-a6ed-af0f894c9c25\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " Jul 10 13:26:31.945281 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945225 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-socket\") pod \"6d271dba-de20-4d38-a6ed-af0f894c9c25\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " Jul 10 13:26:31.945281 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945252 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-envoy\") pod \"6d271dba-de20-4d38-a6ed-af0f894c9c25\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " Jul 10 13:26:31.945427 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945391 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d271dba-de20-4d38-a6ed-af0f894c9c25-istiod-ca-cert\") pod \"6d271dba-de20-4d38-a6ed-af0f894c9c25\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " Jul 10 13:26:31.945486 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945433 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-credential-socket\") pod \"6d271dba-de20-4d38-a6ed-af0f894c9c25\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " Jul 10 13:26:31.945486 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945449 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d271dba-de20-4d38-a6ed-af0f894c9c25" (UID: "6d271dba-de20-4d38-a6ed-af0f894c9c25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:26:31.945486 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945461 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-data\") pod \"6d271dba-de20-4d38-a6ed-af0f894c9c25\" (UID: \"6d271dba-de20-4d38-a6ed-af0f894c9c25\") " Jul 10 13:26:31.945646 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945503 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d271dba-de20-4d38-a6ed-af0f894c9c25" (UID: "6d271dba-de20-4d38-a6ed-af0f894c9c25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:26:31.945748 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945729 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d271dba-de20-4d38-a6ed-af0f894c9c25" (UID: "6d271dba-de20-4d38-a6ed-af0f894c9c25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:26:31.945907 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945890 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-certs\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:26:31.945976 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945913 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-workload-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:26:31.945976 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.945928 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-credential-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:26:31.946084 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.946031 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d271dba-de20-4d38-a6ed-af0f894c9c25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d271dba-de20-4d38-a6ed-af0f894c9c25" (UID: "6d271dba-de20-4d38-a6ed-af0f894c9c25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:26:31.946278 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.946255 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d271dba-de20-4d38-a6ed-af0f894c9c25" (UID: "6d271dba-de20-4d38-a6ed-af0f894c9c25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:26:31.947378 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.947342 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-kube-api-access-snfhr" (OuterVolumeSpecName: "kube-api-access-snfhr") pod "6d271dba-de20-4d38-a6ed-af0f894c9c25" (UID: "6d271dba-de20-4d38-a6ed-af0f894c9c25"). InnerVolumeSpecName "kube-api-access-snfhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:26:31.947378 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.947364 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d271dba-de20-4d38-a6ed-af0f894c9c25" (UID: "6d271dba-de20-4d38-a6ed-af0f894c9c25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:26:31.947512 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.947442 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d271dba-de20-4d38-a6ed-af0f894c9c25" (UID: "6d271dba-de20-4d38-a6ed-af0f894c9c25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:26:31.947599 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:31.947580 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d271dba-de20-4d38-a6ed-af0f894c9c25" (UID: "6d271dba-de20-4d38-a6ed-af0f894c9c25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 13:26:32.047121 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.047088 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d271dba-de20-4d38-a6ed-af0f894c9c25-istiod-ca-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:26:32.047121 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.047118 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-data\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:26:32.047121 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.047126 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-token\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:26:32.047352 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.047134 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-podinfo\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:26:32.047352 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.047142 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snfhr\" (UniqueName: \"kubernetes.io/projected/6d271dba-de20-4d38-a6ed-af0f894c9c25-kube-api-access-snfhr\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:26:32.047352 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.047151 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d271dba-de20-4d38-a6ed-af0f894c9c25-istio-envoy\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:26:32.689095 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.689073 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" Jul 10 13:26:32.689515 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.689116 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j" event={"ID":"6d271dba-de20-4d38-a6ed-af0f894c9c25","Type":"ContainerDied","Data":"f48acf602fd836cb3473d0bd17b8ae5574a50afeceb53f11e3d81a9a44717199"} Jul 10 13:26:32.689515 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.689176 2571 scope.go:117] "RemoveContainer" containerID="d67473539b1e76e9c9ea0246be87e906c032273a2ef433b213dfb7de36282adb" Jul 10 13:26:32.713277 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.713251 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j"] Jul 10 13:26:32.717255 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:32.717223 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-76c01a-openshift-default-bfb894b7-bgt5j"] Jul 10 13:26:33.535542 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:33.535509 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" path="/var/lib/kubelet/pods/6d271dba-de20-4d38-a6ed-af0f894c9c25/volumes" Jul 10 13:26:35.923707 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.923672 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf"] Jul 10 13:26:35.924865 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.924841 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" Jul 10 13:26:35.924865 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.924864 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" Jul 10 13:26:35.925042 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.924878 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" Jul 10 13:26:35.925042 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.924887 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" Jul 10 13:26:35.925042 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.924901 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" Jul 10 13:26:35.925042 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.924909 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" Jul 10 13:26:35.925042 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.925018 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" Jul 10 13:26:35.925042 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.925030 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e52891-24cd-4aeb-8031-3d3af15ad26c" containerName="istio-proxy" Jul 10 13:26:35.925042 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.925041 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" Jul 10 13:26:35.925416 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.925050 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" Jul 10 13:26:35.929854 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.929834 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:35.932848 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.932831 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ed2758-openshift-default-dockercfg-jltmv\"" Jul 10 13:26:35.940370 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.940346 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf"] Jul 10 13:26:35.984487 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.984462 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-envoy\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:35.984598 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.984500 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-data\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:35.984598 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.984523 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-socket\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:35.984726 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.984629 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-credential-socket\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:35.984726 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.984705 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-istio-token\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:35.984834 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.984742 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-certs\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:35.984834 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.984767 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c9aebe-de73-4884-959b-1d19f80ce566-istio-podinfo\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:35.984834 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.984790 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c9aebe-de73-4884-959b-1d19f80ce566-istiod-ca-cert\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:35.984965 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:35.984846 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-246rj\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-kube-api-access-246rj\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086022 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.085986 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-istio-token\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086022 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086029 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-certs\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086250 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086045 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c9aebe-de73-4884-959b-1d19f80ce566-istio-podinfo\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086344 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086244 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c9aebe-de73-4884-959b-1d19f80ce566-istiod-ca-cert\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086409 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086357 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-246rj\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-kube-api-access-246rj\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086484 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086405 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-envoy\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086484 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086433 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-data\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086599 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086564 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-certs\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086599 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086578 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-socket\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086710 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086694 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-credential-socket\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086860 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086836 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-socket\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.086960 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.086942 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-data\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.087034 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.087015 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-credential-socket\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.087091 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.087049 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c9aebe-de73-4884-959b-1d19f80ce566-istiod-ca-cert\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.088613 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.088589 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-envoy\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.088692 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.088675 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c9aebe-de73-4884-959b-1d19f80ce566-istio-podinfo\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.093808 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.093786 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-istio-token\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.093950 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.093934 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-246rj\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-kube-api-access-246rj\") pod \"e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.242222 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.242127 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:36.386069 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.386040 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf"] Jul 10 13:26:36.388018 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:26:36.387990 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c9aebe_de73_4884_959b_1d19f80ce566.slice/crio-ed976f73cc6dd6308640b9cd80dd06d0a600368526bcaa2df018d5d125fcc50b WatchSource:0}: Error finding container ed976f73cc6dd6308640b9cd80dd06d0a600368526bcaa2df018d5d125fcc50b: Status 404 returned error can't find the container with id ed976f73cc6dd6308640b9cd80dd06d0a600368526bcaa2df018d5d125fcc50b Jul 10 13:26:36.390057 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.390025 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:26:36.390124 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.390089 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:26:36.390124 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.390118 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:26:36.717923 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.717888 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" event={"ID":"13c9aebe-de73-4884-959b-1d19f80ce566","Type":"ContainerStarted","Data":"e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1"} Jul 10 13:26:36.718120 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.717949 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" event={"ID":"13c9aebe-de73-4884-959b-1d19f80ce566","Type":"ContainerStarted","Data":"ed976f73cc6dd6308640b9cd80dd06d0a600368526bcaa2df018d5d125fcc50b"} Jul 10 13:26:36.739026 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:36.738971 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" podStartSLOduration=1.73895362 podStartE2EDuration="1.73895362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:26:36.737482066 +0000 UTC m=+2447.623242673" watchObservedRunningTime="2026-07-10 13:26:36.73895362 +0000 UTC m=+2447.624714235" Jul 10 13:26:37.243259 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:37.243226 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:37.247773 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:37.247747 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:37.724664 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:37.724635 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:37.725614 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:37.725590 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:53.766467 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:53.766418 2571 patch_prober.go:28] interesting pod/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 13:26:53.766874 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:53.766480 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 13:26:54.830242 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:54.830197 2571 generic.go:356] "Generic (PLEG): container finished" podID="13c9aebe-de73-4884-959b-1d19f80ce566" containerID="e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1" exitCode=0 Jul 10 13:26:54.830714 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:54.830277 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" event={"ID":"13c9aebe-de73-4884-959b-1d19f80ce566","Type":"ContainerDied","Data":"e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1"} Jul 10 13:26:54.830802 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:54.830744 2571 scope.go:117] "RemoveContainer" containerID="e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1" Jul 10 13:26:54.831822 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:54.831795 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:26:54.831884 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:54.831868 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:26:54.831924 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:54.831901 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:26:55.842198 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:55.842146 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" event={"ID":"13c9aebe-de73-4884-959b-1d19f80ce566","Type":"ContainerStarted","Data":"dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b"} Jul 10 13:26:56.242917 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:56.242870 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:26:56.243217 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:56.242943 2571 patch_prober.go:28] interesting pod/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 10 13:26:56.243416 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:56.243382 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 10 13:26:57.243361 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:57.243322 2571 patch_prober.go:28] interesting pod/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 10 13:26:57.243952 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:57.243391 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 10 13:26:58.243426 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:58.243388 2571 patch_prober.go:28] interesting pod/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 10 13:26:58.244025 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:58.243989 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 10 13:26:59.243319 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:59.242453 2571 patch_prober.go:28] interesting pod/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 10 13:26:59.243319 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:26:59.242511 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 10 13:27:00.246443 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:00.246413 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:27:00.246840 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:00.246815 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:27:00.247448 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:00.247429 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:27:04.965962 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:04.965930 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf"] Jul 10 13:27:04.966373 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:04.966106 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" containerID="cri-o://dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b" gracePeriod=30 Jul 10 13:27:10.232939 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.232879 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:27:10.235061 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235043 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-socket\") pod \"13c9aebe-de73-4884-959b-1d19f80ce566\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " Jul 10 13:27:10.235061 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235068 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c9aebe-de73-4884-959b-1d19f80ce566-istiod-ca-cert\") pod \"13c9aebe-de73-4884-959b-1d19f80ce566\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " Jul 10 13:27:10.235204 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235091 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-certs\") pod \"13c9aebe-de73-4884-959b-1d19f80ce566\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " Jul 10 13:27:10.235246 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235210 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-data\") pod \"13c9aebe-de73-4884-959b-1d19f80ce566\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " Jul 10 13:27:10.235307 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235265 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-istio-token\") pod \"13c9aebe-de73-4884-959b-1d19f80ce566\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " Jul 10 13:27:10.235369 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235309 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-246rj\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-kube-api-access-246rj\") pod \"13c9aebe-de73-4884-959b-1d19f80ce566\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " Jul 10 13:27:10.235369 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235344 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-envoy\") pod \"13c9aebe-de73-4884-959b-1d19f80ce566\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " Jul 10 13:27:10.235475 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235306 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "13c9aebe-de73-4884-959b-1d19f80ce566" (UID: "13c9aebe-de73-4884-959b-1d19f80ce566"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:27:10.235475 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235463 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-credential-socket\") pod \"13c9aebe-de73-4884-959b-1d19f80ce566\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " Jul 10 13:27:10.235580 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235473 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "13c9aebe-de73-4884-959b-1d19f80ce566" (UID: "13c9aebe-de73-4884-959b-1d19f80ce566"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:27:10.235580 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235503 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c9aebe-de73-4884-959b-1d19f80ce566-istio-podinfo\") pod \"13c9aebe-de73-4884-959b-1d19f80ce566\" (UID: \"13c9aebe-de73-4884-959b-1d19f80ce566\") " Jul 10 13:27:10.235580 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235481 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13c9aebe-de73-4884-959b-1d19f80ce566-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "13c9aebe-de73-4884-959b-1d19f80ce566" (UID: "13c9aebe-de73-4884-959b-1d19f80ce566"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:27:10.235740 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235655 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "13c9aebe-de73-4884-959b-1d19f80ce566" (UID: "13c9aebe-de73-4884-959b-1d19f80ce566"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:27:10.235961 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235938 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-credential-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:27:10.235961 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235964 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:27:10.236151 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235982 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/13c9aebe-de73-4884-959b-1d19f80ce566-istiod-ca-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:27:10.236151 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.235995 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-workload-certs\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:27:10.236340 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.236276 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-data" (OuterVolumeSpecName: "istio-data") pod "13c9aebe-de73-4884-959b-1d19f80ce566" (UID: "13c9aebe-de73-4884-959b-1d19f80ce566"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:27:10.237515 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.237487 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-istio-token" (OuterVolumeSpecName: "istio-token") pod "13c9aebe-de73-4884-959b-1d19f80ce566" (UID: "13c9aebe-de73-4884-959b-1d19f80ce566"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:27:10.237618 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.237573 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-kube-api-access-246rj" (OuterVolumeSpecName: "kube-api-access-246rj") pod "13c9aebe-de73-4884-959b-1d19f80ce566" (UID: "13c9aebe-de73-4884-959b-1d19f80ce566"). InnerVolumeSpecName "kube-api-access-246rj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:27:10.237618 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.237582 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "13c9aebe-de73-4884-959b-1d19f80ce566" (UID: "13c9aebe-de73-4884-959b-1d19f80ce566"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:27:10.237699 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.237639 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/13c9aebe-de73-4884-959b-1d19f80ce566-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "13c9aebe-de73-4884-959b-1d19f80ce566" (UID: "13c9aebe-de73-4884-959b-1d19f80ce566"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 13:27:10.337415 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.337381 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-data\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:27:10.337415 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.337404 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-istio-token\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:27:10.337415 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.337413 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-246rj\" (UniqueName: \"kubernetes.io/projected/13c9aebe-de73-4884-959b-1d19f80ce566-kube-api-access-246rj\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:27:10.337415 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.337422 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/13c9aebe-de73-4884-959b-1d19f80ce566-istio-envoy\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:27:10.337680 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.337431 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/13c9aebe-de73-4884-959b-1d19f80ce566-istio-podinfo\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:27:10.937266 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.937226 2571 generic.go:356] "Generic (PLEG): container finished" podID="13c9aebe-de73-4884-959b-1d19f80ce566" containerID="dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b" exitCode=0 Jul 10 13:27:10.937460 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.937270 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" Jul 10 13:27:10.937460 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.937329 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" event={"ID":"13c9aebe-de73-4884-959b-1d19f80ce566","Type":"ContainerDied","Data":"dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b"} Jul 10 13:27:10.937460 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.937386 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf" event={"ID":"13c9aebe-de73-4884-959b-1d19f80ce566","Type":"ContainerDied","Data":"ed976f73cc6dd6308640b9cd80dd06d0a600368526bcaa2df018d5d125fcc50b"} Jul 10 13:27:10.937460 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.937425 2571 scope.go:117] "RemoveContainer" containerID="dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b" Jul 10 13:27:10.947892 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.947875 2571 scope.go:117] "RemoveContainer" containerID="e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1" Jul 10 13:27:10.958134 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.958116 2571 scope.go:117] "RemoveContainer" containerID="dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b" Jul 10 13:27:10.958556 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:27:10.958534 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b\": container with ID starting with dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b not found: ID does not exist" containerID="dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b" Jul 10 13:27:10.958665 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.958563 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b"} err="failed to get container status \"dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b\": rpc error: code = NotFound desc = could not find container \"dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b\": container with ID starting with dae43d83bf933f389c19cdfdba1e848cd1abc53d90e28a5de1d8bb5a0929de8b not found: ID does not exist" Jul 10 13:27:10.958665 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.958587 2571 scope.go:117] "RemoveContainer" containerID="e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1" Jul 10 13:27:10.958937 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:27:10.958904 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1\": container with ID starting with e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1 not found: ID does not exist" containerID="e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1" Jul 10 13:27:10.958991 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.958947 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1"} err="failed to get container status \"e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1\": rpc error: code = NotFound desc = could not find container \"e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1\": container with ID starting with e4efb6e4a7f5beb97567f8b6755f6ae73fe083e4ba4ea817bbb896cd9e3984d1 not found: ID does not exist" Jul 10 13:27:10.962805 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.962785 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf"] Jul 10 13:27:10.967050 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:10.967032 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ed2758-openshift-default-644f9776cf-4grjf"] Jul 10 13:27:11.536112 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:11.536077 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" path="/var/lib/kubelet/pods/13c9aebe-de73-4884-959b-1d19f80ce566/volumes" Jul 10 13:27:13.825244 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.825159 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l"] Jul 10 13:27:13.825708 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.825652 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" Jul 10 13:27:13.825708 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.825664 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" Jul 10 13:27:13.825708 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.825680 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" Jul 10 13:27:13.825708 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.825685 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d271dba-de20-4d38-a6ed-af0f894c9c25" containerName="istio-proxy" Jul 10 13:27:13.825708 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.825693 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" Jul 10 13:27:13.825708 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.825699 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" Jul 10 13:27:13.826032 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.825772 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" Jul 10 13:27:13.826032 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.825781 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="13c9aebe-de73-4884-959b-1d19f80ce566" containerName="istio-proxy" Jul 10 13:27:13.828574 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.828551 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.831419 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.831394 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-d5a846-openshift-default-dockercfg-rswqn\"" Jul 10 13:27:13.839208 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.839188 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l"] Jul 10 13:27:13.873895 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.873866 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-certs\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.874044 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.873906 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-envoy\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.874044 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.873927 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-podinfo\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.874044 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.873945 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-token\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.874044 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.874012 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-data\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.874233 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.874079 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pqtx\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-kube-api-access-9pqtx\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.874233 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.874133 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-socket\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.874233 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.874160 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-credential-socket\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.874233 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.874184 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed9b52a-6303-4778-b1eb-29b269ca2b40-istiod-ca-cert\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.975504 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975472 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-certs\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.975694 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975524 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-envoy\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.975694 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975544 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-podinfo\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.975694 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975677 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-token\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.975878 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975734 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-data\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.975878 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975802 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pqtx\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-kube-api-access-9pqtx\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.975878 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975860 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-socket\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.976032 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975874 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-certs\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.976032 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975893 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-credential-socket\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.976032 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.975921 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed9b52a-6303-4778-b1eb-29b269ca2b40-istiod-ca-cert\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.976193 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.976094 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-data\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.976398 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.976371 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-socket\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.976487 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.976425 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-credential-socket\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.976647 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.976626 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed9b52a-6303-4778-b1eb-29b269ca2b40-istiod-ca-cert\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.977788 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.977765 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-envoy\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.978024 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.978006 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-podinfo\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.983970 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.983946 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-token\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:13.984062 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:13.984041 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pqtx\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-kube-api-access-9pqtx\") pod \"e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:14.140276 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:14.140233 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:14.304481 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:14.304458 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l"] Jul 10 13:27:14.305890 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:27:14.305863 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed9b52a_6303_4778_b1eb_29b269ca2b40.slice/crio-d2f108f3f19b3e3be4f62a16b42d6943e10375e57481e6267b4b61df77c33f0b WatchSource:0}: Error finding container d2f108f3f19b3e3be4f62a16b42d6943e10375e57481e6267b4b61df77c33f0b: Status 404 returned error can't find the container with id d2f108f3f19b3e3be4f62a16b42d6943e10375e57481e6267b4b61df77c33f0b Jul 10 13:27:14.308085 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:14.308051 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:27:14.308191 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:14.308128 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:27:14.308191 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:14.308168 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:27:14.964111 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:14.964075 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" event={"ID":"eed9b52a-6303-4778-b1eb-29b269ca2b40","Type":"ContainerStarted","Data":"fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96"} Jul 10 13:27:14.964521 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:14.964128 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" event={"ID":"eed9b52a-6303-4778-b1eb-29b269ca2b40","Type":"ContainerStarted","Data":"d2f108f3f19b3e3be4f62a16b42d6943e10375e57481e6267b4b61df77c33f0b"} Jul 10 13:27:14.994995 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:14.994949 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" podStartSLOduration=1.994935395 podStartE2EDuration="1.994935395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:27:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:27:14.992524947 +0000 UTC m=+2485.878285563" watchObservedRunningTime="2026-07-10 13:27:14.994935395 +0000 UTC m=+2485.880696010" Jul 10 13:27:15.140974 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:15.140947 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:15.145694 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:15.145670 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:15.970153 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:15.970125 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:15.970981 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:15.970961 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:32.012458 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:32.012413 2571 patch_prober.go:28] interesting pod/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 13:27:32.014236 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:32.012502 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 13:27:32.074636 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:32.074606 2571 generic.go:356] "Generic (PLEG): container finished" podID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerID="fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96" exitCode=0 Jul 10 13:27:32.074764 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:32.074681 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" event={"ID":"eed9b52a-6303-4778-b1eb-29b269ca2b40","Type":"ContainerDied","Data":"fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96"} Jul 10 13:27:32.075145 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:32.075132 2571 scope.go:117] "RemoveContainer" containerID="fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96" Jul 10 13:27:32.076249 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:32.076222 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:27:32.076347 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:32.076329 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:27:32.076395 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:32.076366 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 10 13:27:33.082611 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:33.082582 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" event={"ID":"eed9b52a-6303-4778-b1eb-29b269ca2b40","Type":"ContainerStarted","Data":"b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63"} Jul 10 13:27:33.140790 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:33.140403 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:33.141067 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:33.140854 2571 patch_prober.go:28] interesting pod/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 10 13:27:33.141228 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:33.141205 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 10 13:27:34.141093 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:34.141049 2571 patch_prober.go:28] interesting pod/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 10 13:27:34.141608 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:34.141133 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 10 13:27:35.141130 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:35.141094 2571 patch_prober.go:28] interesting pod/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 10 13:27:35.141725 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:35.141692 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 10 13:27:36.141389 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:36.141348 2571 patch_prober.go:28] interesting pod/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 10 13:27:36.141889 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:36.141434 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 10 13:27:37.144119 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:37.144092 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:37.144530 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:37.144443 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:27:37.145064 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:27:37.145047 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:30:00.134742 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.134696 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h"] Jul 10 13:30:00.137640 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.137614 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" Jul 10 13:30:00.140217 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.140197 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sfczk\"" Jul 10 13:30:00.140505 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.140492 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np"] Jul 10 13:30:00.142960 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.142924 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" Jul 10 13:30:00.143246 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.143219 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq"] Jul 10 13:30:00.146800 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.146524 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" Jul 10 13:30:00.147219 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.147197 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h"] Jul 10 13:30:00.150437 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.150419 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" Jul 10 13:30:00.158051 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.158030 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np"] Jul 10 13:30:00.161365 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.161343 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq"] Jul 10 13:30:00.161628 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.161607 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" Jul 10 13:30:00.202236 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.202215 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" Jul 10 13:30:00.328215 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.328174 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np"] Jul 10 13:30:00.333157 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:30:00.332344 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed6f4d9_044c_419f_8e0c_bd08b88c6cdf.slice/crio-6144ab17dfb7db75c55081d994302b45a0aa1a4b856c3633ced378be1d0d287d WatchSource:0}: Error finding container 6144ab17dfb7db75c55081d994302b45a0aa1a4b856c3633ced378be1d0d287d: Status 404 returned error can't find the container with id 6144ab17dfb7db75c55081d994302b45a0aa1a4b856c3633ced378be1d0d287d Jul 10 13:30:00.384880 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.384816 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq"] Jul 10 13:30:00.386216 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:30:00.386196 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ce978c_fcd0_4919_925e_6311bff2f4be.slice/crio-cbf0cde8875aff6c83071273e6046fdf7eed8432eb66e14451a430b7d06986ec WatchSource:0}: Error finding container cbf0cde8875aff6c83071273e6046fdf7eed8432eb66e14451a430b7d06986ec: Status 404 returned error can't find the container with id cbf0cde8875aff6c83071273e6046fdf7eed8432eb66e14451a430b7d06986ec Jul 10 13:30:00.549779 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.549755 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h"] Jul 10 13:30:00.620940 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:30:00.620910 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5850042d_dba3_48f5_9e15_fe0260ed4978.slice/crio-f245f4652314187fda62f17db1fb2f47468d7094fe3256a7609eb2fbddbbac63 WatchSource:0}: Error finding container f245f4652314187fda62f17db1fb2f47468d7094fe3256a7609eb2fbddbbac63: Status 404 returned error can't find the container with id f245f4652314187fda62f17db1fb2f47468d7094fe3256a7609eb2fbddbbac63 Jul 10 13:30:00.951790 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.951762 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" event={"ID":"64ce978c-fcd0-4919-925e-6311bff2f4be","Type":"ContainerStarted","Data":"6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a"} Jul 10 13:30:00.951910 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.951823 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" event={"ID":"64ce978c-fcd0-4919-925e-6311bff2f4be","Type":"ContainerStarted","Data":"cbf0cde8875aff6c83071273e6046fdf7eed8432eb66e14451a430b7d06986ec"} Jul 10 13:30:00.953409 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.953384 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" event={"ID":"5850042d-dba3-48f5-9e15-fe0260ed4978","Type":"ContainerStarted","Data":"038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f"} Jul 10 13:30:00.953513 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.953432 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" event={"ID":"5850042d-dba3-48f5-9e15-fe0260ed4978","Type":"ContainerStarted","Data":"f245f4652314187fda62f17db1fb2f47468d7094fe3256a7609eb2fbddbbac63"} Jul 10 13:30:00.954810 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.954785 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" event={"ID":"9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf","Type":"ContainerStarted","Data":"33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364"} Jul 10 13:30:00.954895 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.954834 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" event={"ID":"9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf","Type":"ContainerStarted","Data":"6144ab17dfb7db75c55081d994302b45a0aa1a4b856c3633ced378be1d0d287d"} Jul 10 13:30:00.987599 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:00.987558 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" podStartSLOduration=0.987545236 podStartE2EDuration="987.545236ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:30:00.985531543 +0000 UTC m=+2651.871292157" watchObservedRunningTime="2026-07-10 13:30:00.987545236 +0000 UTC m=+2651.873305851" Jul 10 13:30:01.017030 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:01.016991 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" podStartSLOduration=1.01697806 podStartE2EDuration="1.01697806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:30:01.014876617 +0000 UTC m=+2651.900637234" watchObservedRunningTime="2026-07-10 13:30:01.01697806 +0000 UTC m=+2651.902738675" Jul 10 13:30:01.036549 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:30:01.036501 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" podStartSLOduration=1.036489864 podStartE2EDuration="1.036489864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:30:01.033265207 +0000 UTC m=+2651.919025822" watchObservedRunningTime="2026-07-10 13:30:01.036489864 +0000 UTC m=+2651.922250479" Jul 10 13:31:00.011635 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:00.011531 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np"] Jul 10 13:31:00.012386 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:00.011911 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" podUID="9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf" containerName="cleanup" containerID="cri-o://33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364" gracePeriod=30 Jul 10 13:31:00.015144 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:00.015121 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq"] Jul 10 13:31:00.015350 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:00.015327 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" podUID="64ce978c-fcd0-4919-925e-6311bff2f4be" containerName="cleanup" containerID="cri-o://6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a" gracePeriod=30 Jul 10 13:31:00.017881 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:00.017859 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h"] Jul 10 13:31:00.018020 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:00.018002 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" podUID="5850042d-dba3-48f5-9e15-fe0260ed4978" containerName="cleanup" containerID="cri-o://038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f" gracePeriod=30 Jul 10 13:31:30.217565 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.217538 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" Jul 10 13:31:30.280081 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.280056 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" Jul 10 13:31:30.283572 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.283552 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" Jul 10 13:31:30.494412 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.494380 2571 generic.go:356] "Generic (PLEG): container finished" podID="64ce978c-fcd0-4919-925e-6311bff2f4be" containerID="6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a" exitCode=137 Jul 10 13:31:30.494593 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.494432 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" Jul 10 13:31:30.494593 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.494445 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" event={"ID":"64ce978c-fcd0-4919-925e-6311bff2f4be","Type":"ContainerDied","Data":"6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a"} Jul 10 13:31:30.494593 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.494503 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq" event={"ID":"64ce978c-fcd0-4919-925e-6311bff2f4be","Type":"ContainerDied","Data":"cbf0cde8875aff6c83071273e6046fdf7eed8432eb66e14451a430b7d06986ec"} Jul 10 13:31:30.494593 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.494534 2571 scope.go:117] "RemoveContainer" containerID="6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a" Jul 10 13:31:30.495879 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.495858 2571 generic.go:356] "Generic (PLEG): container finished" podID="5850042d-dba3-48f5-9e15-fe0260ed4978" containerID="038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f" exitCode=137 Jul 10 13:31:30.495976 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.495896 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" Jul 10 13:31:30.495976 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.495938 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" event={"ID":"5850042d-dba3-48f5-9e15-fe0260ed4978","Type":"ContainerDied","Data":"038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f"} Jul 10 13:31:30.495976 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.495974 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h" event={"ID":"5850042d-dba3-48f5-9e15-fe0260ed4978","Type":"ContainerDied","Data":"f245f4652314187fda62f17db1fb2f47468d7094fe3256a7609eb2fbddbbac63"} Jul 10 13:31:30.497683 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.497657 2571 generic.go:356] "Generic (PLEG): container finished" podID="9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf" containerID="33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364" exitCode=137 Jul 10 13:31:30.497924 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.497708 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" Jul 10 13:31:30.498038 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.497725 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" event={"ID":"9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf","Type":"ContainerDied","Data":"33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364"} Jul 10 13:31:30.498110 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.498029 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np" event={"ID":"9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf","Type":"ContainerDied","Data":"6144ab17dfb7db75c55081d994302b45a0aa1a4b856c3633ced378be1d0d287d"} Jul 10 13:31:30.504496 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.504478 2571 scope.go:117] "RemoveContainer" containerID="6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a" Jul 10 13:31:30.504767 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:31:30.504748 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a\": container with ID starting with 6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a not found: ID does not exist" containerID="6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a" Jul 10 13:31:30.504834 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.504776 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a"} err="failed to get container status \"6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a\": rpc error: code = NotFound desc = could not find container \"6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a\": container with ID starting with 6ee0b5f19fee719285d181211dccc90fe1f37786ed3705ea02cf1e1d0c7ace4a not found: ID does not exist" Jul 10 13:31:30.504834 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.504799 2571 scope.go:117] "RemoveContainer" containerID="038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f" Jul 10 13:31:30.513792 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.513776 2571 scope.go:117] "RemoveContainer" containerID="038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f" Jul 10 13:31:30.514026 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:31:30.514007 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f\": container with ID starting with 038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f not found: ID does not exist" containerID="038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f" Jul 10 13:31:30.514073 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.514033 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f"} err="failed to get container status \"038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f\": rpc error: code = NotFound desc = could not find container \"038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f\": container with ID starting with 038a3560cd6d11943aa92f782b969ee5488f00984e10a6af8d671378b1b24e5f not found: ID does not exist" Jul 10 13:31:30.514073 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.514050 2571 scope.go:117] "RemoveContainer" containerID="33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364" Jul 10 13:31:30.523774 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.523750 2571 scope.go:117] "RemoveContainer" containerID="33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364" Jul 10 13:31:30.524053 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:31:30.524030 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364\": container with ID starting with 33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364 not found: ID does not exist" containerID="33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364" Jul 10 13:31:30.524120 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.524063 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364"} err="failed to get container status \"33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364\": rpc error: code = NotFound desc = could not find container \"33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364\": container with ID starting with 33aa0487540dd35eb9219a03f2e21e831ec0ee0f09e1814faafb26e747a62364 not found: ID does not exist" Jul 10 13:31:30.524615 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.524597 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h"] Jul 10 13:31:30.527704 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.527684 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d5a846-29728170-9vv9h"] Jul 10 13:31:30.536663 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.536643 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np"] Jul 10 13:31:30.540610 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.540590 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728170-m75np"] Jul 10 13:31:30.551478 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.551458 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq"] Jul 10 13:31:30.556965 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:30.556946 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7266ad-29728170-tbbwq"] Jul 10 13:31:31.536867 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:31.536836 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5850042d-dba3-48f5-9e15-fe0260ed4978" path="/var/lib/kubelet/pods/5850042d-dba3-48f5-9e15-fe0260ed4978/volumes" Jul 10 13:31:31.537237 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:31.537096 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64ce978c-fcd0-4919-925e-6311bff2f4be" path="/var/lib/kubelet/pods/64ce978c-fcd0-4919-925e-6311bff2f4be/volumes" Jul 10 13:31:31.537320 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:31:31.537307 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf" path="/var/lib/kubelet/pods/9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf/volumes" Jul 10 13:35:35.373115 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373072 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:35.373758 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373738 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf" containerName="cleanup" Jul 10 13:35:35.373846 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373762 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf" containerName="cleanup" Jul 10 13:35:35.373846 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373791 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5850042d-dba3-48f5-9e15-fe0260ed4978" containerName="cleanup" Jul 10 13:35:35.373846 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373799 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5850042d-dba3-48f5-9e15-fe0260ed4978" containerName="cleanup" Jul 10 13:35:35.373846 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373830 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ce978c-fcd0-4919-925e-6311bff2f4be" containerName="cleanup" Jul 10 13:35:35.373846 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373839 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ce978c-fcd0-4919-925e-6311bff2f4be" containerName="cleanup" Jul 10 13:35:35.374098 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373949 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="64ce978c-fcd0-4919-925e-6311bff2f4be" containerName="cleanup" Jul 10 13:35:35.374098 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373962 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed6f4d9-044c-419f-8e0c-bd08b88c6cdf" containerName="cleanup" Jul 10 13:35:35.374098 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.373972 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5850042d-dba3-48f5-9e15-fe0260ed4978" containerName="cleanup" Jul 10 13:35:35.376792 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.376772 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:35.379507 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.379482 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-8s9xp\"" Jul 10 13:35:35.380717 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.380692 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:35.492147 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.492112 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpn7l\" (UniqueName: \"kubernetes.io/projected/5a90c3e8-f7f0-42d2-a194-61104ee5efd0-kube-api-access-hpn7l\") pod \"test-curl-7376-140175642517344\" (UID: \"5a90c3e8-f7f0-42d2-a194-61104ee5efd0\") " pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:35.593383 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.593341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpn7l\" (UniqueName: \"kubernetes.io/projected/5a90c3e8-f7f0-42d2-a194-61104ee5efd0-kube-api-access-hpn7l\") pod \"test-curl-7376-140175642517344\" (UID: \"5a90c3e8-f7f0-42d2-a194-61104ee5efd0\") " pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:35.601946 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.601919 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpn7l\" (UniqueName: \"kubernetes.io/projected/5a90c3e8-f7f0-42d2-a194-61104ee5efd0-kube-api-access-hpn7l\") pod \"test-curl-7376-140175642517344\" (UID: \"5a90c3e8-f7f0-42d2-a194-61104ee5efd0\") " pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:35.687999 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:35.687966 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:36.030101 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:36.030012 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:36.036160 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:35:36.036134 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a90c3e8_f7f0_42d2_a194_61104ee5efd0.slice/crio-f3b1b9a9c56f379e0184679083c7b38efa4e9b65e0163c79891144f42e929aa8 WatchSource:0}: Error finding container f3b1b9a9c56f379e0184679083c7b38efa4e9b65e0163c79891144f42e929aa8: Status 404 returned error can't find the container with id f3b1b9a9c56f379e0184679083c7b38efa4e9b65e0163c79891144f42e929aa8 Jul 10 13:35:36.037409 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:36.037393 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 13:35:36.960391 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:36.960286 2571 generic.go:356] "Generic (PLEG): container finished" podID="5a90c3e8-f7f0-42d2-a194-61104ee5efd0" containerID="3435832e5d3ad742c780e68d9050ccc8321cfdbfb572d7375b72945736da9605" exitCode=0 Jul 10 13:35:36.960738 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:36.960387 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7376-140175642517344" event={"ID":"5a90c3e8-f7f0-42d2-a194-61104ee5efd0","Type":"ContainerDied","Data":"3435832e5d3ad742c780e68d9050ccc8321cfdbfb572d7375b72945736da9605"} Jul 10 13:35:36.960738 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:36.960426 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7376-140175642517344" event={"ID":"5a90c3e8-f7f0-42d2-a194-61104ee5efd0","Type":"ContainerStarted","Data":"f3b1b9a9c56f379e0184679083c7b38efa4e9b65e0163c79891144f42e929aa8"} Jul 10 13:35:38.106300 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.106264 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:38.142090 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.142052 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7376-140175642517344_5a90c3e8-f7f0-42d2-a194-61104ee5efd0/test-curl-7376-140175642517344/0.log" Jul 10 13:35:38.169042 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.169020 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:38.172372 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.172350 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:38.220876 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.220819 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpn7l\" (UniqueName: \"kubernetes.io/projected/5a90c3e8-f7f0-42d2-a194-61104ee5efd0-kube-api-access-hpn7l\") pod \"5a90c3e8-f7f0-42d2-a194-61104ee5efd0\" (UID: \"5a90c3e8-f7f0-42d2-a194-61104ee5efd0\") " Jul 10 13:35:38.222757 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.222731 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a90c3e8-f7f0-42d2-a194-61104ee5efd0-kube-api-access-hpn7l" (OuterVolumeSpecName: "kube-api-access-hpn7l") pod "5a90c3e8-f7f0-42d2-a194-61104ee5efd0" (UID: "5a90c3e8-f7f0-42d2-a194-61104ee5efd0"). InnerVolumeSpecName "kube-api-access-hpn7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:35:38.306537 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.306510 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:38.306988 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.306975 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a90c3e8-f7f0-42d2-a194-61104ee5efd0" containerName="test-curl-7376-140175642517344" Jul 10 13:35:38.307040 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.306989 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a90c3e8-f7f0-42d2-a194-61104ee5efd0" containerName="test-curl-7376-140175642517344" Jul 10 13:35:38.307083 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.307071 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a90c3e8-f7f0-42d2-a194-61104ee5efd0" containerName="test-curl-7376-140175642517344" Jul 10 13:35:38.310165 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.310151 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:38.314565 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.314544 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:38.322016 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.321996 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpn7l\" (UniqueName: \"kubernetes.io/projected/5a90c3e8-f7f0-42d2-a194-61104ee5efd0-kube-api-access-hpn7l\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:35:38.422837 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.422802 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppz2j\" (UniqueName: \"kubernetes.io/projected/06ccf40e-b60d-4242-8cef-8ca4952ae6ab-kube-api-access-ppz2j\") pod \"test-curl-7376-140175642517344\" (UID: \"06ccf40e-b60d-4242-8cef-8ca4952ae6ab\") " pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:38.523863 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.523778 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppz2j\" (UniqueName: \"kubernetes.io/projected/06ccf40e-b60d-4242-8cef-8ca4952ae6ab-kube-api-access-ppz2j\") pod \"test-curl-7376-140175642517344\" (UID: \"06ccf40e-b60d-4242-8cef-8ca4952ae6ab\") " pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:38.533025 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.532997 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppz2j\" (UniqueName: \"kubernetes.io/projected/06ccf40e-b60d-4242-8cef-8ca4952ae6ab-kube-api-access-ppz2j\") pod \"test-curl-7376-140175642517344\" (UID: \"06ccf40e-b60d-4242-8cef-8ca4952ae6ab\") " pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:38.620421 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.620392 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:38.754074 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.754048 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:38.755753 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:35:38.755718 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06ccf40e_b60d_4242_8cef_8ca4952ae6ab.slice/crio-e1e81b3e64cdd7319736a4af3f040a8ad1ee871e4a3af6bdeaf6cef3b18b3fab WatchSource:0}: Error finding container e1e81b3e64cdd7319736a4af3f040a8ad1ee871e4a3af6bdeaf6cef3b18b3fab: Status 404 returned error can't find the container with id e1e81b3e64cdd7319736a4af3f040a8ad1ee871e4a3af6bdeaf6cef3b18b3fab Jul 10 13:35:38.973041 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.973011 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7376-140175642517344" event={"ID":"06ccf40e-b60d-4242-8cef-8ca4952ae6ab","Type":"ContainerStarted","Data":"e1e81b3e64cdd7319736a4af3f040a8ad1ee871e4a3af6bdeaf6cef3b18b3fab"} Jul 10 13:35:38.974242 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.974227 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3b1b9a9c56f379e0184679083c7b38efa4e9b65e0163c79891144f42e929aa8" Jul 10 13:35:38.974332 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.974241 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:38.986277 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:38.986251 2571 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7376-140175642517344" oldPodUID="5a90c3e8-f7f0-42d2-a194-61104ee5efd0" podUID="06ccf40e-b60d-4242-8cef-8ca4952ae6ab" Jul 10 13:35:39.537234 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:39.537205 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a90c3e8-f7f0-42d2-a194-61104ee5efd0" path="/var/lib/kubelet/pods/5a90c3e8-f7f0-42d2-a194-61104ee5efd0/volumes" Jul 10 13:35:39.981742 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:39.981706 2571 generic.go:356] "Generic (PLEG): container finished" podID="06ccf40e-b60d-4242-8cef-8ca4952ae6ab" containerID="e7f1ffb210f4dd0fe0e62cdd415b90ee9f1117c4b866dcc910b43b065df283d8" exitCode=0 Jul 10 13:35:39.981903 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:39.981796 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7376-140175642517344" event={"ID":"06ccf40e-b60d-4242-8cef-8ca4952ae6ab","Type":"ContainerDied","Data":"e7f1ffb210f4dd0fe0e62cdd415b90ee9f1117c4b866dcc910b43b065df283d8"} Jul 10 13:35:41.140953 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.140928 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:41.161001 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.160974 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7376-140175642517344_06ccf40e-b60d-4242-8cef-8ca4952ae6ab/test-curl-7376-140175642517344/0.log" Jul 10 13:35:41.184712 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.184682 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:41.187235 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.187213 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7376-140175642517344"] Jul 10 13:35:41.257351 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.257268 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppz2j\" (UniqueName: \"kubernetes.io/projected/06ccf40e-b60d-4242-8cef-8ca4952ae6ab-kube-api-access-ppz2j\") pod \"06ccf40e-b60d-4242-8cef-8ca4952ae6ab\" (UID: \"06ccf40e-b60d-4242-8cef-8ca4952ae6ab\") " Jul 10 13:35:41.259259 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.259231 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06ccf40e-b60d-4242-8cef-8ca4952ae6ab-kube-api-access-ppz2j" (OuterVolumeSpecName: "kube-api-access-ppz2j") pod "06ccf40e-b60d-4242-8cef-8ca4952ae6ab" (UID: "06ccf40e-b60d-4242-8cef-8ca4952ae6ab"). InnerVolumeSpecName "kube-api-access-ppz2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:35:41.307603 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.307578 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7376-140175642516192"] Jul 10 13:35:41.308047 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.308034 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06ccf40e-b60d-4242-8cef-8ca4952ae6ab" containerName="test-curl-7376-140175642517344" Jul 10 13:35:41.308100 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.308048 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="06ccf40e-b60d-4242-8cef-8ca4952ae6ab" containerName="test-curl-7376-140175642517344" Jul 10 13:35:41.308145 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.308133 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="06ccf40e-b60d-4242-8cef-8ca4952ae6ab" containerName="test-curl-7376-140175642517344" Jul 10 13:35:41.311355 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.311340 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642516192" Jul 10 13:35:41.317238 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.317214 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7376-140175642516192"] Jul 10 13:35:41.358190 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.358167 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppz2j\" (UniqueName: \"kubernetes.io/projected/06ccf40e-b60d-4242-8cef-8ca4952ae6ab-kube-api-access-ppz2j\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:35:41.459329 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.459302 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vbwj\" (UniqueName: \"kubernetes.io/projected/4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345-kube-api-access-5vbwj\") pod \"test-curl-7376-140175642516192\" (UID: \"4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345\") " pod="openshift-ingress/test-curl-7376-140175642516192" Jul 10 13:35:41.536912 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.536840 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06ccf40e-b60d-4242-8cef-8ca4952ae6ab" path="/var/lib/kubelet/pods/06ccf40e-b60d-4242-8cef-8ca4952ae6ab/volumes" Jul 10 13:35:41.560135 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.560105 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vbwj\" (UniqueName: \"kubernetes.io/projected/4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345-kube-api-access-5vbwj\") pod \"test-curl-7376-140175642516192\" (UID: \"4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345\") " pod="openshift-ingress/test-curl-7376-140175642516192" Jul 10 13:35:41.568281 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.568261 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vbwj\" (UniqueName: \"kubernetes.io/projected/4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345-kube-api-access-5vbwj\") pod \"test-curl-7376-140175642516192\" (UID: \"4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345\") " pod="openshift-ingress/test-curl-7376-140175642516192" Jul 10 13:35:41.622038 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.622018 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642516192" Jul 10 13:35:41.757915 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.757892 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7376-140175642516192"] Jul 10 13:35:41.758922 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:35:41.758896 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ddb36eb_44cd_4eb3_a0bb_24cd2ed30345.slice/crio-96b0e5f554c11aca1ef4f27a79881391b1a1b56a302aef3755df2c0052d763b7 WatchSource:0}: Error finding container 96b0e5f554c11aca1ef4f27a79881391b1a1b56a302aef3755df2c0052d763b7: Status 404 returned error can't find the container with id 96b0e5f554c11aca1ef4f27a79881391b1a1b56a302aef3755df2c0052d763b7 Jul 10 13:35:41.996020 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.995991 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642517344" Jul 10 13:35:41.996020 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.996005 2571 scope.go:117] "RemoveContainer" containerID="e7f1ffb210f4dd0fe0e62cdd415b90ee9f1117c4b866dcc910b43b065df283d8" Jul 10 13:35:41.997564 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:41.997534 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7376-140175642516192" event={"ID":"4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345","Type":"ContainerStarted","Data":"96b0e5f554c11aca1ef4f27a79881391b1a1b56a302aef3755df2c0052d763b7"} Jul 10 13:35:43.006366 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:43.006279 2571 generic.go:356] "Generic (PLEG): container finished" podID="4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345" containerID="e0422bbe276e8dbb45c396ac0373f9d1becb198120d0eaa6d745c60de7bc653d" exitCode=0 Jul 10 13:35:43.006766 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:43.006368 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7376-140175642516192" event={"ID":"4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345","Type":"ContainerDied","Data":"e0422bbe276e8dbb45c396ac0373f9d1becb198120d0eaa6d745c60de7bc653d"} Jul 10 13:35:44.164831 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:44.164809 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642516192" Jul 10 13:35:44.184109 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:44.184082 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7376-140175642516192_4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345/test-curl-7376-140175642516192/0.log" Jul 10 13:35:44.210790 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:44.210763 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7376-140175642516192"] Jul 10 13:35:44.213322 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:44.213300 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7376-140175642516192"] Jul 10 13:35:44.286240 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:44.286170 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vbwj\" (UniqueName: \"kubernetes.io/projected/4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345-kube-api-access-5vbwj\") pod \"4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345\" (UID: \"4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345\") " Jul 10 13:35:44.288183 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:44.288161 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345-kube-api-access-5vbwj" (OuterVolumeSpecName: "kube-api-access-5vbwj") pod "4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345" (UID: "4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345"). InnerVolumeSpecName "kube-api-access-5vbwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:35:44.387085 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:44.387053 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vbwj\" (UniqueName: \"kubernetes.io/projected/4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345-kube-api-access-5vbwj\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:35:45.021246 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:45.021219 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7376-140175642516192" Jul 10 13:35:45.021506 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:45.021266 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96b0e5f554c11aca1ef4f27a79881391b1a1b56a302aef3755df2c0052d763b7" Jul 10 13:35:45.031258 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:35:45.031229 2571 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7376-140175642516192\" is forbidden: User \"system:node:ip-10-0-143-162.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-162.ec2.internal' and this object" podUID="4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345" pod="openshift-ingress/test-curl-7376-140175642516192" Jul 10 13:35:45.535397 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:35:45.535359 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345" path="/var/lib/kubelet/pods/4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345/volumes" Jul 10 13:36:01.314487 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:01.314447 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l"] Jul 10 13:36:01.314891 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:01.314713 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" containerID="cri-o://b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63" gracePeriod=30 Jul 10 13:36:01.989844 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:01.989805 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56986cdcfc-tvxh8_cdc8816a-0460-4f96-80e7-78bf79157b23/authorino/0.log" Jul 10 13:36:06.579284 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.579261 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:36:06.604594 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604519 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-data\") pod \"eed9b52a-6303-4778-b1eb-29b269ca2b40\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " Jul 10 13:36:06.604594 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604556 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-envoy\") pod \"eed9b52a-6303-4778-b1eb-29b269ca2b40\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " Jul 10 13:36:06.604594 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604583 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed9b52a-6303-4778-b1eb-29b269ca2b40-istiod-ca-cert\") pod \"eed9b52a-6303-4778-b1eb-29b269ca2b40\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " Jul 10 13:36:06.604839 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604663 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-socket\") pod \"eed9b52a-6303-4778-b1eb-29b269ca2b40\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " Jul 10 13:36:06.604839 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604690 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-podinfo\") pod \"eed9b52a-6303-4778-b1eb-29b269ca2b40\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " Jul 10 13:36:06.604839 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604719 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-token\") pod \"eed9b52a-6303-4778-b1eb-29b269ca2b40\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " Jul 10 13:36:06.604839 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604791 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-certs\") pod \"eed9b52a-6303-4778-b1eb-29b269ca2b40\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " Jul 10 13:36:06.604839 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604830 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-credential-socket\") pod \"eed9b52a-6303-4778-b1eb-29b269ca2b40\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " Jul 10 13:36:06.605088 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604885 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pqtx\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-kube-api-access-9pqtx\") pod \"eed9b52a-6303-4778-b1eb-29b269ca2b40\" (UID: \"eed9b52a-6303-4778-b1eb-29b269ca2b40\") " Jul 10 13:36:06.605088 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.604999 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eed9b52a-6303-4778-b1eb-29b269ca2b40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eed9b52a-6303-4778-b1eb-29b269ca2b40" (UID: "eed9b52a-6303-4778-b1eb-29b269ca2b40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 13:36:06.605208 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.605181 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eed9b52a-6303-4778-b1eb-29b269ca2b40" (UID: "eed9b52a-6303-4778-b1eb-29b269ca2b40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:36:06.605377 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.605352 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:36:06.605377 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.605376 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eed9b52a-6303-4778-b1eb-29b269ca2b40-istiod-ca-cert\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:36:06.605533 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.605398 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eed9b52a-6303-4778-b1eb-29b269ca2b40" (UID: "eed9b52a-6303-4778-b1eb-29b269ca2b40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:36:06.605533 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.605432 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-data" (OuterVolumeSpecName: "istio-data") pod "eed9b52a-6303-4778-b1eb-29b269ca2b40" (UID: "eed9b52a-6303-4778-b1eb-29b269ca2b40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:36:06.605646 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.605551 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eed9b52a-6303-4778-b1eb-29b269ca2b40" (UID: "eed9b52a-6303-4778-b1eb-29b269ca2b40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:36:06.607213 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.607184 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eed9b52a-6303-4778-b1eb-29b269ca2b40" (UID: "eed9b52a-6303-4778-b1eb-29b269ca2b40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 13:36:06.607331 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.607248 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-kube-api-access-9pqtx" (OuterVolumeSpecName: "kube-api-access-9pqtx") pod "eed9b52a-6303-4778-b1eb-29b269ca2b40" (UID: "eed9b52a-6303-4778-b1eb-29b269ca2b40"). InnerVolumeSpecName "kube-api-access-9pqtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:36:06.607569 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.607546 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-token" (OuterVolumeSpecName: "istio-token") pod "eed9b52a-6303-4778-b1eb-29b269ca2b40" (UID: "eed9b52a-6303-4778-b1eb-29b269ca2b40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 13:36:06.607639 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.607548 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eed9b52a-6303-4778-b1eb-29b269ca2b40" (UID: "eed9b52a-6303-4778-b1eb-29b269ca2b40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 13:36:06.705916 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.705880 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-podinfo\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:36:06.705916 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.705911 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-token\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:36:06.706106 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.705925 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-workload-certs\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:36:06.706106 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.705933 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-credential-socket\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:36:06.706106 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.705947 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pqtx\" (UniqueName: \"kubernetes.io/projected/eed9b52a-6303-4778-b1eb-29b269ca2b40-kube-api-access-9pqtx\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:36:06.706106 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.705968 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-data\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:36:06.706106 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.705979 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eed9b52a-6303-4778-b1eb-29b269ca2b40-istio-envoy\") on node \"ip-10-0-143-162.ec2.internal\" DevicePath \"\"" Jul 10 13:36:06.891492 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:06.891455 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lsdvj_67976ed9-c2b3-4983-8196-f22ec66af6bd/manager/0.log" Jul 10 13:36:07.005591 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.005566 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xcvtc_8dc0227f-b22b-4207-828a-415efd0d2b9d/manager/0.log" Jul 10 13:36:07.155449 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.155364 2571 generic.go:356] "Generic (PLEG): container finished" podID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerID="b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63" exitCode=0 Jul 10 13:36:07.155449 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.155420 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" Jul 10 13:36:07.155637 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.155454 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" event={"ID":"eed9b52a-6303-4778-b1eb-29b269ca2b40","Type":"ContainerDied","Data":"b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63"} Jul 10 13:36:07.155637 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.155533 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l" event={"ID":"eed9b52a-6303-4778-b1eb-29b269ca2b40","Type":"ContainerDied","Data":"d2f108f3f19b3e3be4f62a16b42d6943e10375e57481e6267b4b61df77c33f0b"} Jul 10 13:36:07.155637 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.155556 2571 scope.go:117] "RemoveContainer" containerID="b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63" Jul 10 13:36:07.165386 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.165367 2571 scope.go:117] "RemoveContainer" containerID="fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96" Jul 10 13:36:07.174565 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.174548 2571 scope.go:117] "RemoveContainer" containerID="b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63" Jul 10 13:36:07.174882 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:36:07.174839 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63\": container with ID starting with b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63 not found: ID does not exist" containerID="b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63" Jul 10 13:36:07.174985 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.174895 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63"} err="failed to get container status \"b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63\": rpc error: code = NotFound desc = could not find container \"b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63\": container with ID starting with b5e6c2e10610b15913621cfedd33edc507f22a0379dbf4787c432b43bddf6d63 not found: ID does not exist" Jul 10 13:36:07.174985 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.174941 2571 scope.go:117] "RemoveContainer" containerID="fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96" Jul 10 13:36:07.175284 ip-10-0-143-162 kubenswrapper[2571]: E0710 13:36:07.175252 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96\": container with ID starting with fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96 not found: ID does not exist" containerID="fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96" Jul 10 13:36:07.175284 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.175282 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96"} err="failed to get container status \"fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96\": rpc error: code = NotFound desc = could not find container \"fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96\": container with ID starting with fca6eb0f95a2cfe8619141fe48fc9090df313a15bb20c6f5543dd7e6f57ead96 not found: ID does not exist" Jul 10 13:36:07.178633 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.178614 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l"] Jul 10 13:36:07.182110 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.182089 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-d5a846-openshift-default-698b9cc9c5-tq68l"] Jul 10 13:36:07.248227 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.248204 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dfc9h_36c309c1-45e5-479f-93b2-51fd699e86f2/server/0.log" Jul 10 13:36:07.365837 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.365803 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5q4nh_eae18245-0cf2-4dac-af9e-1c1a7972d7e0/manager/0.log" Jul 10 13:36:07.482815 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.482743 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6798dbc-44fgl_85c83e63-b8ee-4e12-ab12-de17359db348/copy-manifests/0.log" Jul 10 13:36:07.490304 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.490259 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6798dbc-44fgl_85c83e63-b8ee-4e12-ab12-de17359db348/manager/0.log" Jul 10 13:36:07.535436 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:07.535404 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" path="/var/lib/kubelet/pods/eed9b52a-6303-4778-b1eb-29b269ca2b40/volumes" Jul 10 13:36:08.965171 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:08.965142 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9_c077e5c6-39f9-4c23-a616-b6a1e3516aad/util/0.log" Jul 10 13:36:08.970913 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:08.970882 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9_c077e5c6-39f9-4c23-a616-b6a1e3516aad/pull/0.log" Jul 10 13:36:08.976229 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:08.976202 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9_c077e5c6-39f9-4c23-a616-b6a1e3516aad/extract/0.log" Jul 10 13:36:09.087523 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.087489 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n_ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f/util/0.log" Jul 10 13:36:09.094333 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.094305 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n_ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f/pull/0.log" Jul 10 13:36:09.100354 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.100333 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n_ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f/extract/0.log" Jul 10 13:36:09.216671 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.216588 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf_9ea73571-bdb4-443f-9188-2608105f000a/util/0.log" Jul 10 13:36:09.223121 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.223096 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf_9ea73571-bdb4-443f-9188-2608105f000a/pull/0.log" Jul 10 13:36:09.233105 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.233077 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf_9ea73571-bdb4-443f-9188-2608105f000a/extract/0.log" Jul 10 13:36:09.341655 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.341622 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk_794c8a39-094d-45a2-b5fa-48da12241ccc/util/0.log" Jul 10 13:36:09.348466 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.348437 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk_794c8a39-094d-45a2-b5fa-48da12241ccc/pull/0.log" Jul 10 13:36:09.354873 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.354855 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk_794c8a39-094d-45a2-b5fa-48da12241ccc/extract/0.log" Jul 10 13:36:09.481937 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.481865 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56986cdcfc-tvxh8_cdc8816a-0460-4f96-80e7-78bf79157b23/authorino/0.log" Jul 10 13:36:09.604567 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.604544 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-frszx_4a1159c1-17bb-4050-99cf-bf53f01744bc/manager/0.log" Jul 10 13:36:09.712593 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.712549 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-pnqd9_fdc4f54b-a816-4e00-8624-ac3ba69fdc39/manager/0.log" Jul 10 13:36:09.941956 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:09.941921 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-j9wqv_f32cef55-66bb-4a77-be3a-aaeb6084bcc8/registry-server/0.log" Jul 10 13:36:10.130103 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:10.130067 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zvz69_e912041a-26ec-4ce5-8bc3-84f54997b916/manager/0.log" Jul 10 13:36:10.380078 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:10.380002 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fx7tb_3786890e-a8b9-4bbd-b1d9-e2354d8898b5/manager/0.log" Jul 10 13:36:10.769967 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:10.769803 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-695cd9447dsp57g_fbd98ae2-6a89-4ad5-8614-96ea5f071f72/istio-proxy/0.log" Jul 10 13:36:11.147161 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:11.147135 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779ffb979b-xnq4n_d111e575-84e5-4a66-9401-c0beefbb3176/kube-auth-proxy/0.log" Jul 10 13:36:11.267387 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:11.267361 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5df9b4f89-n5d84_0a12194a-bafd-421a-b2ed-7c24f8b2f292/istio-proxy/1.log" Jul 10 13:36:11.384686 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:11.384658 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-f92z7_4773c3e1-c9ef-489a-bc27-7843f5aec186/payload-pre-processing/0.log" Jul 10 13:36:11.632116 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:11.632046 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794657f478-2bsbk_77b92f38-8541-4c6c-b440-e05f745570ec/router/0.log" Jul 10 13:36:11.991054 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:11.991023 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-glp98_6edb48aa-5ea8-423a-9c1c-477d762bb07d/main/0.log" Jul 10 13:36:16.155520 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.155484 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qthfs/must-gather-blzw7"] Jul 10 13:36:16.156035 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.155932 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" Jul 10 13:36:16.156035 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.155945 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" Jul 10 13:36:16.156035 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.155958 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" Jul 10 13:36:16.156035 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.155964 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" Jul 10 13:36:16.156035 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.155973 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345" containerName="test-curl-7376-140175642516192" Jul 10 13:36:16.156035 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.155979 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345" containerName="test-curl-7376-140175642516192" Jul 10 13:36:16.156253 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.156046 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ddb36eb-44cd-4eb3-a0bb-24cd2ed30345" containerName="test-curl-7376-140175642516192" Jul 10 13:36:16.156253 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.156055 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" Jul 10 13:36:16.156253 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.156063 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="eed9b52a-6303-4778-b1eb-29b269ca2b40" containerName="istio-proxy" Jul 10 13:36:16.160783 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.160764 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qthfs/must-gather-blzw7" Jul 10 13:36:16.168641 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.168615 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qthfs\"/\"default-dockercfg-x4wbf\"" Jul 10 13:36:16.168775 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.168654 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qthfs\"/\"kube-root-ca.crt\"" Jul 10 13:36:16.169509 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.169488 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qthfs\"/\"openshift-service-ca.crt\"" Jul 10 13:36:16.172558 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.172539 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qthfs/must-gather-blzw7"] Jul 10 13:36:16.284896 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.284868 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7b5w\" (UniqueName: \"kubernetes.io/projected/ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b-kube-api-access-k7b5w\") pod \"must-gather-blzw7\" (UID: \"ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b\") " pod="openshift-must-gather-qthfs/must-gather-blzw7" Jul 10 13:36:16.285056 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.284916 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b-must-gather-output\") pod \"must-gather-blzw7\" (UID: \"ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b\") " pod="openshift-must-gather-qthfs/must-gather-blzw7" Jul 10 13:36:16.385488 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.385454 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7b5w\" (UniqueName: \"kubernetes.io/projected/ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b-kube-api-access-k7b5w\") pod \"must-gather-blzw7\" (UID: \"ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b\") " pod="openshift-must-gather-qthfs/must-gather-blzw7" Jul 10 13:36:16.385653 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.385504 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b-must-gather-output\") pod \"must-gather-blzw7\" (UID: \"ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b\") " pod="openshift-must-gather-qthfs/must-gather-blzw7" Jul 10 13:36:16.385769 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.385755 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b-must-gather-output\") pod \"must-gather-blzw7\" (UID: \"ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b\") " pod="openshift-must-gather-qthfs/must-gather-blzw7" Jul 10 13:36:16.398566 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.398542 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7b5w\" (UniqueName: \"kubernetes.io/projected/ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b-kube-api-access-k7b5w\") pod \"must-gather-blzw7\" (UID: \"ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b\") " pod="openshift-must-gather-qthfs/must-gather-blzw7" Jul 10 13:36:16.470276 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.470208 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qthfs/must-gather-blzw7" Jul 10 13:36:16.640088 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:16.640062 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qthfs/must-gather-blzw7"] Jul 10 13:36:16.642005 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:36:16.641979 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce86e42e_165f_4a11_82cb_6bbe9d9bfe6b.slice/crio-c894d6e8a19f920b6657a8f590a38fa597b5053a2fb3d200ae215adc2fd3556e WatchSource:0}: Error finding container c894d6e8a19f920b6657a8f590a38fa597b5053a2fb3d200ae215adc2fd3556e: Status 404 returned error can't find the container with id c894d6e8a19f920b6657a8f590a38fa597b5053a2fb3d200ae215adc2fd3556e Jul 10 13:36:17.220950 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:17.220919 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qthfs/must-gather-blzw7" event={"ID":"ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b","Type":"ContainerStarted","Data":"c894d6e8a19f920b6657a8f590a38fa597b5053a2fb3d200ae215adc2fd3556e"} Jul 10 13:36:18.231451 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:18.231406 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qthfs/must-gather-blzw7" event={"ID":"ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b","Type":"ContainerStarted","Data":"fdb074af096aad50425814ea5e56c6d90cd981ed41009ca7b81385d69fc5ccf4"} Jul 10 13:36:18.231944 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:18.231471 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qthfs/must-gather-blzw7" event={"ID":"ce86e42e-165f-4a11-82cb-6bbe9d9bfe6b","Type":"ContainerStarted","Data":"8dc20d3386df18572d6722f540e57b0722cd7292a1d188b075747d4abdaaa5db"} Jul 10 13:36:18.258022 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:18.257971 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qthfs/must-gather-blzw7" podStartSLOduration=1.306920923 podStartE2EDuration="2.257956443s" totalImagesPullingTime="951.03552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 13:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:36:18.255693305 +0000 UTC m=+3029.141453921" watchObservedRunningTime="2026-07-10 13:36:18.257956443 +0000 UTC m=+3029.143717058" Jul 10 13:36:19.185927 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:19.185892 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8ln2b_ea5247d2-85e6-4959-9479-a6eea1212a2d/global-pull-secret-syncer/0.log" Jul 10 13:36:19.371430 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:19.371395 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gmfsc_83fab052-d823-4c52-bf03-26323aba154d/konnectivity-agent/0.log" Jul 10 13:36:19.500523 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:19.500434 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-162.ec2.internal_d1078cbfb33567ae55c66f9d84943893/haproxy/0.log" Jul 10 13:36:23.492393 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.492367 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9_c077e5c6-39f9-4c23-a616-b6a1e3516aad/extract/0.log" Jul 10 13:36:23.520917 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.520874 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9_c077e5c6-39f9-4c23-a616-b6a1e3516aad/util/0.log" Jul 10 13:36:23.562604 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.562335 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759bggn9_c077e5c6-39f9-4c23-a616-b6a1e3516aad/pull/0.log" Jul 10 13:36:23.609045 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.608769 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n_ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f/extract/0.log" Jul 10 13:36:23.640495 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.640465 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n_ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f/util/0.log" Jul 10 13:36:23.661562 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.661487 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0lb64n_ad99d4fa-8e2a-4804-b7e6-d33c458d8f8f/pull/0.log" Jul 10 13:36:23.691428 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.691391 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf_9ea73571-bdb4-443f-9188-2608105f000a/extract/0.log" Jul 10 13:36:23.718828 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.718801 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf_9ea73571-bdb4-443f-9188-2608105f000a/util/0.log" Jul 10 13:36:23.742164 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.742133 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jnvpf_9ea73571-bdb4-443f-9188-2608105f000a/pull/0.log" Jul 10 13:36:23.780706 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.780675 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk_794c8a39-094d-45a2-b5fa-48da12241ccc/extract/0.log" Jul 10 13:36:23.802872 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.802848 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk_794c8a39-094d-45a2-b5fa-48da12241ccc/util/0.log" Jul 10 13:36:23.835849 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:23.835824 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19g2zk_794c8a39-094d-45a2-b5fa-48da12241ccc/pull/0.log" Jul 10 13:36:24.395355 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:24.395324 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56986cdcfc-tvxh8_cdc8816a-0460-4f96-80e7-78bf79157b23/authorino/0.log" Jul 10 13:36:24.446441 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:24.446393 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-frszx_4a1159c1-17bb-4050-99cf-bf53f01744bc/manager/0.log" Jul 10 13:36:24.490118 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:24.490071 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-pnqd9_fdc4f54b-a816-4e00-8624-ac3ba69fdc39/manager/0.log" Jul 10 13:36:24.609968 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:24.609014 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-j9wqv_f32cef55-66bb-4a77-be3a-aaeb6084bcc8/registry-server/0.log" Jul 10 13:36:24.829166 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:24.827700 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zvz69_e912041a-26ec-4ce5-8bc3-84f54997b916/manager/0.log" Jul 10 13:36:24.984316 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:24.983824 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fx7tb_3786890e-a8b9-4bbd-b1d9-e2354d8898b5/manager/0.log" Jul 10 13:36:26.577232 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.577137 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb9dce00-6e9c-4ffe-90ce-91f300552654/alertmanager/0.log" Jul 10 13:36:26.628676 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.628645 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb9dce00-6e9c-4ffe-90ce-91f300552654/config-reloader/0.log" Jul 10 13:36:26.683310 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.680510 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb9dce00-6e9c-4ffe-90ce-91f300552654/kube-rbac-proxy-web/0.log" Jul 10 13:36:26.733201 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.733165 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb9dce00-6e9c-4ffe-90ce-91f300552654/kube-rbac-proxy/0.log" Jul 10 13:36:26.774191 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.774155 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb9dce00-6e9c-4ffe-90ce-91f300552654/kube-rbac-proxy-metric/0.log" Jul 10 13:36:26.810354 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.810270 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb9dce00-6e9c-4ffe-90ce-91f300552654/prom-label-proxy/0.log" Jul 10 13:36:26.840538 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.840510 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb9dce00-6e9c-4ffe-90ce-91f300552654/init-config-reloader/0.log" Jul 10 13:36:26.924592 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.924556 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-hfmlp_5abfdd0d-eebd-4f32-9d6d-04497a2f2bed/kube-state-metrics/0.log" Jul 10 13:36:26.951797 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.951763 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-hfmlp_5abfdd0d-eebd-4f32-9d6d-04497a2f2bed/kube-rbac-proxy-main/0.log" Jul 10 13:36:26.985216 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:26.985189 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-hfmlp_5abfdd0d-eebd-4f32-9d6d-04497a2f2bed/kube-rbac-proxy-self/0.log" Jul 10 13:36:27.323846 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.323810 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rkv8j_5c5b5c42-b040-42a0-8d7d-074af02ae819/node-exporter/0.log" Jul 10 13:36:27.350139 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.350109 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rkv8j_5c5b5c42-b040-42a0-8d7d-074af02ae819/kube-rbac-proxy/0.log" Jul 10 13:36:27.374730 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.374697 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rkv8j_5c5b5c42-b040-42a0-8d7d-074af02ae819/init-textfile/0.log" Jul 10 13:36:27.422073 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.422040 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-pftgv_2270b672-089c-4691-a432-c7a20ef79918/kube-rbac-proxy-main/0.log" Jul 10 13:36:27.449818 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.449789 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-pftgv_2270b672-089c-4691-a432-c7a20ef79918/kube-rbac-proxy-self/0.log" Jul 10 13:36:27.477378 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.477346 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-pftgv_2270b672-089c-4691-a432-c7a20ef79918/openshift-state-metrics/0.log" Jul 10 13:36:27.766458 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.766409 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rd6rk_0f225da2-9370-456b-84c5-a5f65bf83660/prometheus-operator/0.log" Jul 10 13:36:27.790105 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.790065 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rd6rk_0f225da2-9370-456b-84c5-a5f65bf83660/kube-rbac-proxy/0.log" Jul 10 13:36:27.821111 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.821076 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-6hv98_b0e2fa4c-7ed3-431c-b438-cacc3838ff8b/prometheus-operator-admission-webhook/0.log" Jul 10 13:36:27.861330 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.861303 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5dfdc7d66d-srdpd_4ee8cb5a-53aa-4894-a754-0cb6ecd988e8/telemeter-client/0.log" Jul 10 13:36:27.890840 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.890816 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5dfdc7d66d-srdpd_4ee8cb5a-53aa-4894-a754-0cb6ecd988e8/reload/0.log" Jul 10 13:36:27.917170 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.917129 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5dfdc7d66d-srdpd_4ee8cb5a-53aa-4894-a754-0cb6ecd988e8/kube-rbac-proxy/0.log" Jul 10 13:36:27.957682 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.957657 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fc8ff5d5-5ddz4_009b7f1e-d869-4bc8-a21c-fdfe21ec7d78/thanos-query/0.log" Jul 10 13:36:27.984764 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:27.984740 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fc8ff5d5-5ddz4_009b7f1e-d869-4bc8-a21c-fdfe21ec7d78/kube-rbac-proxy-web/0.log" Jul 10 13:36:28.011691 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:28.011663 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fc8ff5d5-5ddz4_009b7f1e-d869-4bc8-a21c-fdfe21ec7d78/kube-rbac-proxy/0.log" Jul 10 13:36:28.038740 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:28.038712 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fc8ff5d5-5ddz4_009b7f1e-d869-4bc8-a21c-fdfe21ec7d78/prom-label-proxy/0.log" Jul 10 13:36:28.065062 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:28.065032 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fc8ff5d5-5ddz4_009b7f1e-d869-4bc8-a21c-fdfe21ec7d78/kube-rbac-proxy-rules/0.log" Jul 10 13:36:28.100688 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:28.100655 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8fc8ff5d5-5ddz4_009b7f1e-d869-4bc8-a21c-fdfe21ec7d78/kube-rbac-proxy-metrics/0.log" Jul 10 13:36:29.223261 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.223230 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-xzc8p_00b769db-9621-4725-8449-04103e7c94ec/networking-console-plugin/0.log" Jul 10 13:36:29.507739 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.507652 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68"] Jul 10 13:36:29.513763 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.513739 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.521893 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.521550 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68"] Jul 10 13:36:29.648270 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.648227 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-proc\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.648511 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.648487 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-lib-modules\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.648583 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.648544 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgg57\" (UniqueName: \"kubernetes.io/projected/05eedafa-38f0-4371-991e-df121c027995-kube-api-access-vgg57\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.648641 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.648608 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-podres\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.648704 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.648640 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-sys\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.750008 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.749962 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-lib-modules\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.750192 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.750026 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgg57\" (UniqueName: \"kubernetes.io/projected/05eedafa-38f0-4371-991e-df121c027995-kube-api-access-vgg57\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.750192 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.750076 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-podres\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.750192 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.750103 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-sys\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.750192 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.750173 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-proc\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.750547 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.750353 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-proc\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.750547 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.750488 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-lib-modules\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.750926 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.750900 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-podres\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.751015 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.750971 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/05eedafa-38f0-4371-991e-df121c027995-sys\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.762510 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.762439 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgg57\" (UniqueName: \"kubernetes.io/projected/05eedafa-38f0-4371-991e-df121c027995-kube-api-access-vgg57\") pod \"perf-node-gather-daemonset-rhk68\" (UID: \"05eedafa-38f0-4371-991e-df121c027995\") " pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:29.851867 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:29.851824 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:30.039895 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:30.039869 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68"] Jul 10 13:36:30.044360 ip-10-0-143-162 kubenswrapper[2571]: W0710 13:36:30.044326 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod05eedafa_38f0_4371_991e_df121c027995.slice/crio-cc083c8c81c101008c4e7e4c8dac20dc9816199cf0b92c44b6ac7394666fa688 WatchSource:0}: Error finding container cc083c8c81c101008c4e7e4c8dac20dc9816199cf0b92c44b6ac7394666fa688: Status 404 returned error can't find the container with id cc083c8c81c101008c4e7e4c8dac20dc9816199cf0b92c44b6ac7394666fa688 Jul 10 13:36:30.354381 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:30.354254 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" event={"ID":"05eedafa-38f0-4371-991e-df121c027995","Type":"ContainerStarted","Data":"fa0d45e3dfad75834cdd3cf9c481900f40e20aa1830b1e9e4589a85050422e33"} Jul 10 13:36:30.354381 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:30.354337 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" event={"ID":"05eedafa-38f0-4371-991e-df121c027995","Type":"ContainerStarted","Data":"cc083c8c81c101008c4e7e4c8dac20dc9816199cf0b92c44b6ac7394666fa688"} Jul 10 13:36:30.355013 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:30.354386 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:30.374384 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:30.374317 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" podStartSLOduration=1.374279423 podStartE2EDuration="1.374279423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 13:36:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 13:36:30.370937236 +0000 UTC m=+3041.256697851" watchObservedRunningTime="2026-07-10 13:36:30.374279423 +0000 UTC m=+3041.260040039" Jul 10 13:36:30.444152 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:30.444122 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78864954cb-gj78d_4e1ead29-c08c-4f7c-b2b6-b5dcc1bcbf35/console/0.log" Jul 10 13:36:30.491539 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:30.491498 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-xsnvv_af33c8f8-7312-4ff0-a28f-153f8f48e5d8/download-server/0.log" Jul 10 13:36:32.153027 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:32.152991 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p7l2z_0694c98e-3ce3-45cb-8503-47b1acd27007/dns/0.log" Jul 10 13:36:32.174404 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:32.174382 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p7l2z_0694c98e-3ce3-45cb-8503-47b1acd27007/kube-rbac-proxy/0.log" Jul 10 13:36:32.243854 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:32.243825 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6hl8t_4aeb355e-62c8-4c45-9b4a-795ab74cfe42/dns-node-resolver/0.log" Jul 10 13:36:32.885700 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:32.885665 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-84kdj_390a4dd4-04cc-4db6-93e5-33ee6222c93e/node-ca/0.log" Jul 10 13:36:33.853935 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:33.853896 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-695cd9447dsp57g_fbd98ae2-6a89-4ad5-8614-96ea5f071f72/istio-proxy/0.log" Jul 10 13:36:34.133451 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:34.133362 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779ffb979b-xnq4n_d111e575-84e5-4a66-9401-c0beefbb3176/kube-auth-proxy/0.log" Jul 10 13:36:34.165128 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:34.165099 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5df9b4f89-n5d84_0a12194a-bafd-421a-b2ed-7c24f8b2f292/istio-proxy/0.log" Jul 10 13:36:34.232719 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:34.232694 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5df9b4f89-n5d84_0a12194a-bafd-421a-b2ed-7c24f8b2f292/istio-proxy/1.log" Jul 10 13:36:34.272519 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:34.272488 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-f92z7_4773c3e1-c9ef-489a-bc27-7843f5aec186/payload-pre-processing/0.log" Jul 10 13:36:34.342517 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:34.342482 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794657f478-2bsbk_77b92f38-8541-4c6c-b440-e05f745570ec/router/0.log" Jul 10 13:36:35.049609 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:35.049578 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prpw8_4f2f42e7-8190-41ef-a9a6-ec8b351dc05e/serve-healthcheck-canary/0.log" Jul 10 13:36:35.646522 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:35.646493 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mrqt7_49dc2bdb-3536-46a3-9dff-3483bd8aa4e1/kube-rbac-proxy/0.log" Jul 10 13:36:35.667592 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:35.667555 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mrqt7_49dc2bdb-3536-46a3-9dff-3483bd8aa4e1/exporter/0.log" Jul 10 13:36:35.688562 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:35.688526 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mrqt7_49dc2bdb-3536-46a3-9dff-3483bd8aa4e1/extractor/0.log" Jul 10 13:36:36.377458 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:36.377432 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qthfs/perf-node-gather-daemonset-rhk68" Jul 10 13:36:37.939471 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:37.939441 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lsdvj_67976ed9-c2b3-4983-8196-f22ec66af6bd/manager/0.log" Jul 10 13:36:38.248993 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:38.248450 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xcvtc_8dc0227f-b22b-4207-828a-415efd0d2b9d/manager/0.log" Jul 10 13:36:38.467135 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:38.467106 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dfc9h_36c309c1-45e5-479f-93b2-51fd699e86f2/server/0.log" Jul 10 13:36:38.636771 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:38.636734 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5q4nh_eae18245-0cf2-4dac-af9e-1c1a7972d7e0/manager/0.log" Jul 10 13:36:38.692546 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:38.692511 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6798dbc-44fgl_85c83e63-b8ee-4e12-ab12-de17359db348/manager/0.log" Jul 10 13:36:38.719095 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:38.719070 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6798dbc-44fgl_85c83e63-b8ee-4e12-ab12-de17359db348/copy-manifests/0.log" Jul 10 13:36:39.524567 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:39.524513 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-695cd9447dsp57g_fbd98ae2-6a89-4ad5-8614-96ea5f071f72/istio-proxy/0.log" Jul 10 13:36:39.783535 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:39.783501 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779ffb979b-xnq4n_d111e575-84e5-4a66-9401-c0beefbb3176/kube-auth-proxy/0.log" Jul 10 13:36:39.804163 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:39.804129 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5df9b4f89-n5d84_0a12194a-bafd-421a-b2ed-7c24f8b2f292/istio-proxy/0.log" Jul 10 13:36:39.872666 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:39.872638 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5df9b4f89-n5d84_0a12194a-bafd-421a-b2ed-7c24f8b2f292/istio-proxy/1.log" Jul 10 13:36:39.907362 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:39.907258 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-f92z7_4773c3e1-c9ef-489a-bc27-7843f5aec186/payload-pre-processing/0.log" Jul 10 13:36:39.962387 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:39.962353 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794657f478-2bsbk_77b92f38-8541-4c6c-b440-e05f745570ec/router/0.log" Jul 10 13:36:41.164894 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:41.164803 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-695cd9447dsp57g_fbd98ae2-6a89-4ad5-8614-96ea5f071f72/istio-proxy/0.log" Jul 10 13:36:41.428419 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:41.428332 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-779ffb979b-xnq4n_d111e575-84e5-4a66-9401-c0beefbb3176/kube-auth-proxy/0.log" Jul 10 13:36:41.448513 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:41.448483 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5df9b4f89-n5d84_0a12194a-bafd-421a-b2ed-7c24f8b2f292/istio-proxy/0.log" Jul 10 13:36:41.505086 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:41.505048 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5df9b4f89-n5d84_0a12194a-bafd-421a-b2ed-7c24f8b2f292/istio-proxy/1.log" Jul 10 13:36:41.539925 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:41.539892 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-f92z7_4773c3e1-c9ef-489a-bc27-7843f5aec186/payload-pre-processing/0.log" Jul 10 13:36:41.593887 ip-10-0-143-162 kubenswrapper[2571]: I0710 13:36:41.593861 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-794657f478-2bsbk_77b92f38-8541-4c6c-b440-e05f745570ec/router/0.log"