Jul 06 22:18:09.127806 ip-10-0-142-114 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 22:18:09.127819 ip-10-0-142-114 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 22:18:09.127827 ip-10-0-142-114 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 22:18:09.128086 ip-10-0-142-114 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 22:18:19.212574 ip-10-0-142-114 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 22:18:19.212591 ip-10-0-142-114 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9354760fa9f9455781c65a2da01066fe -- Jul 06 22:20:08.670902 ip-10-0-142-114 systemd[1]: Starting Kubernetes Kubelet... Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.016114 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020686 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020702 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020705 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020709 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020712 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020715 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020718 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:20:10.276577 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020721 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:20:10.220464 ip-10-0-142-114 systemd[1]: Started Kubernetes Kubelet. Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020723 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020726 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020731 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020734 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020742 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020745 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020748 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020751 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020754 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020757 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020760 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020763 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020766 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020768 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020771 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020775 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020778 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020781 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020784 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:20:10.277877 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020787 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020790 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020793 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020796 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020798 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020801 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020803 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020806 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020808 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020811 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020813 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020817 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020820 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020823 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020825 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020828 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020832 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020835 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020839 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:20:10.278628 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020842 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020845 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020847 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020850 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020853 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020856 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020859 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020861 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020864 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020867 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020870 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020872 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020875 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020878 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020880 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020883 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020886 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020889 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020891 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:20:10.279256 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020894 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020896 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020899 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020902 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020904 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020907 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020910 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020913 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020915 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020918 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020920 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020923 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020926 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020929 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020931 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020934 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020936 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020941 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020943 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020946 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:20:10.279939 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020948 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020951 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020953 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020956 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020959 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020961 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020964 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020966 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020969 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020972 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020975 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020977 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020980 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.020982 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021738 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021746 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021750 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021753 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021756 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021759 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:20:10.280912 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021763 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021765 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021768 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021770 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021773 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021775 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021778 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021781 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021784 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021786 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021789 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021792 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021794 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021797 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021800 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021802 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021805 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021807 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021810 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:20:10.281915 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021813 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021816 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021818 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021822 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021826 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021830 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021833 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021835 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021838 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021841 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021855 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021858 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021861 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021865 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021868 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021871 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021875 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021877 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021880 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:20:10.282883 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021883 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021885 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021888 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021891 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021894 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021897 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021899 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021902 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021905 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021908 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021911 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021914 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021917 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021920 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021922 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021924 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021927 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021929 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021932 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021934 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021936 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:20:10.283677 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021939 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021942 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021944 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021947 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021949 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021952 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021955 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021958 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021961 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021964 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021966 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021969 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021971 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021974 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021977 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021980 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021982 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021986 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021989 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:20:10.284388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021991 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021994 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021996 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.021998 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022001 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022003 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022006 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022008 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022010 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022014 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022017 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022020 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022022 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022025 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022027 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022419 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022425 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022428 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022431 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:20:10.284976 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022434 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022437 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022439 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022442 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022444 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022447 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022450 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022452 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022456 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022459 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022462 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022464 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022467 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022469 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022472 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022474 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022477 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022479 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022482 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022485 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:20:10.285570 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022487 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022489 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022492 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022495 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022497 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022500 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022503 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022506 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022508 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022511 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022515 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022519 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022522 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022525 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022528 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022531 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022533 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022536 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022538 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:20:10.286248 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022541 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022543 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022546 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022548 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022551 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022554 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022556 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022559 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022561 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022564 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022567 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022570 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022588 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022590 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022593 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022596 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022599 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022603 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022606 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022608 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:20:10.286757 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022612 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022615 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022618 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022620 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022623 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022626 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022628 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022633 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022636 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022639 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022642 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022645 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022648 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022650 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022653 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022656 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022658 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022661 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022664 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:20:10.287346 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022683 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022686 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022689 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022692 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022695 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022699 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022702 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022704 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022710 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022713 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022715 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022718 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022721 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022724 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022726 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022729 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.022731 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022803 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022810 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022817 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022821 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 22:20:10.287970 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022826 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022830 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022834 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022838 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022841 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022844 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022848 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022851 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022854 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022857 2578 flags.go:64] FLAG: --cgroup-root="" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022860 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022863 2578 flags.go:64] FLAG: --client-ca-file="" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022866 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022869 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022874 2578 flags.go:64] FLAG: --cluster-domain="" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022876 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022879 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022883 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022886 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022893 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022896 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022899 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022902 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022905 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 06 22:20:10.288804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022908 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022911 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022914 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022917 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022922 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022925 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022928 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022931 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022934 2578 flags.go:64] FLAG: --enable-server="true" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022937 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022942 2578 flags.go:64] FLAG: --event-burst="100" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022945 2578 flags.go:64] FLAG: --event-qps="50" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022948 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022951 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022954 2578 flags.go:64] FLAG: --eviction-hard="" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022958 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022961 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022964 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022967 2578 flags.go:64] FLAG: --eviction-soft="" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022970 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022973 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022976 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022978 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022981 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022985 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 22:20:10.289659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022988 2578 flags.go:64] FLAG: --feature-gates="" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022992 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.022996 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023000 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023003 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023006 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023008 2578 flags.go:64] FLAG: --help="false" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023012 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023015 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023018 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023021 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023024 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023028 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023031 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023034 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023037 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023040 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023043 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023047 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023050 2578 flags.go:64] FLAG: --kube-reserved="" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023052 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023055 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023058 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023061 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 22:20:10.290534 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023064 2578 flags.go:64] FLAG: --lock-file="" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023067 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023070 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023073 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023078 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023081 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023083 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023087 2578 flags.go:64] FLAG: --logging-format="text" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023089 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023093 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023097 2578 flags.go:64] FLAG: --manifest-url="" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023100 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023104 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023107 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023114 2578 flags.go:64] FLAG: --max-pods="110" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023117 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023120 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023122 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023126 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023136 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023139 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023142 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023150 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023153 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023156 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 22:20:10.293455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023159 2578 flags.go:64] FLAG: --pod-cidr="" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023162 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023168 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023171 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023174 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023177 2578 flags.go:64] FLAG: --port="10250" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023180 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023183 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03b435a171c173012" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023186 2578 flags.go:64] FLAG: --qos-reserved="" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023189 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023192 2578 flags.go:64] FLAG: --register-node="true" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023194 2578 flags.go:64] FLAG: --register-with-taints="" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023198 2578 flags.go:64] FLAG: --registry-burst="10" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023201 2578 flags.go:64] FLAG: --registry-qps="5" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023204 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023206 2578 flags.go:64] FLAG: --reserved-memory="" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023210 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023213 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023217 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023220 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023223 2578 flags.go:64] FLAG: --runonce="false" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023227 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023230 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023233 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023236 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 22:20:10.294287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023239 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023243 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023246 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023248 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023251 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023254 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023257 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023260 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023264 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023267 2578 flags.go:64] FLAG: --system-cgroups="" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023270 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023275 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023278 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023281 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023284 2578 flags.go:64] FLAG: --tls-min-version="" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023287 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023290 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023292 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023295 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023298 2578 flags.go:64] FLAG: --v="2" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023302 2578 flags.go:64] FLAG: --version="false" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023306 2578 flags.go:64] FLAG: --vmodule="" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023309 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.023313 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023428 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:20:10.506352 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023434 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023437 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023440 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023444 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023447 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023449 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023452 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023455 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023457 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023460 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023463 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023465 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023468 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023470 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023473 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023475 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023478 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023481 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023484 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023486 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:20:10.529232 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023489 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023492 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023494 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023497 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023499 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023502 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023504 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023507 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023509 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023512 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023514 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023517 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023521 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023524 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023527 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023531 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023533 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023536 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023539 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023541 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:20:10.529986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023546 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023549 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023552 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023554 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023557 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023559 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023562 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023565 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023567 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023570 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023587 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023590 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023592 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023595 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023598 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023600 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023604 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023608 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023611 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:20:10.530576 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023614 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023616 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023619 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023622 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023624 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023628 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023631 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023634 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023638 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023640 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023643 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023645 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023648 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023652 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023656 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023659 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023661 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023665 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023667 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:20:10.531527 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023671 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023674 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023676 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023679 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023681 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023684 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023687 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023690 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023692 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023695 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023697 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023700 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023702 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023705 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023707 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023710 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023713 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023715 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023720 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:20:10.532090 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.023722 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.024214 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.033963 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.033981 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.034605 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.034617 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034676 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034682 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034686 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034689 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034692 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034695 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034697 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034700 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034702 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:20:10.532728 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034705 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034708 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034710 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034713 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034715 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034718 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034720 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034723 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034725 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034728 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034730 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034733 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034736 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034738 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034741 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034744 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034747 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034750 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034758 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034761 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:20:10.533187 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034764 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034766 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034769 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034772 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034776 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034780 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034783 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034786 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034788 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034791 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034793 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034796 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034799 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034802 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034804 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034807 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034810 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034812 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034815 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034818 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:20:10.533803 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034820 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034823 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034825 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034828 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034831 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034834 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034837 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034839 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034844 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034847 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034849 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034852 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034855 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034858 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034860 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034863 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034867 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034870 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034873 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:20:10.773403 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034877 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034880 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034883 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034886 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034889 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034892 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034895 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034897 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034900 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034904 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034907 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034911 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034914 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034917 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034920 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034923 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034925 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034928 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034931 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034933 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:20:10.774359 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034936 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034940 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034943 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034945 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034948 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034950 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034953 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034955 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034958 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034962 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.034964 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.034970 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035089 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035095 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:20:10.774986 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035100 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035103 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035106 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035109 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035112 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035116 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035119 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035122 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035124 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035127 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035139 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035143 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035146 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035148 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035151 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035154 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035156 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035159 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035161 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:20:10.775541 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035166 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035168 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035171 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035173 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035176 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035178 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035181 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035184 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035186 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035190 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035193 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035195 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035198 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035201 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035203 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035206 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035208 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035211 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035213 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035216 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:20:10.776164 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035219 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035221 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035223 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035226 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035230 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035233 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035235 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035238 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035240 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035243 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035245 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035248 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035253 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035256 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035259 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035262 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035265 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035267 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035270 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:20:10.776756 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035272 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035275 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035278 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035281 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035284 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035286 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035289 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035291 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035294 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035296 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035298 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035301 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035303 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035306 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035308 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035311 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035313 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035316 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035319 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035321 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:20:10.777496 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035324 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035326 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035329 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035331 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035334 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035337 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035340 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035342 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035345 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035347 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035350 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035352 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035355 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035358 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035361 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035364 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035366 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035369 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:20:10.778136 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:09.035372 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.035377 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.035508 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.037556 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.038674 2578 server.go:1019] "Starting client certificate rotation" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.038775 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.038808 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.040744 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.042007 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.050704 2578 log.go:25] "Validated CRI v1 runtime API" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.054483 2578 log.go:25] "Validated CRI v1 image API" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.056739 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 22:20:10.778760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.058820 2578 fs.go:135] Filesystem UUIDs: map[27b9bb92-d143-4a1e-98b7-25fe58b34609:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9816a34b-a3fd-4433-b2b7-0155568f0218:/dev/nvme0n1p3] Jul 06 22:20:10.779185 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:09.058835 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 22:20:10.779371 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.197336 2578 manager.go:216] Machine: {Timestamp:2026-07-06 22:20:09.063687481 +0000 UTC m=+0.297949961 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100412 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec220852d1d65704792dfc11b201a797 SystemUUID:ec220852-d1d6-5704-792d-fc11b201a797 BootID:9354760f-a9f9-4557-81c6-5a2da01066fe Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4e:6b:87:0c:ff Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4e:6b:87:0c:ff Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:74:c9:08:ff:d5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 22:20:10.779371 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.197448 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 22:20:10.779371 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.197540 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 22:20:10.779371 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.199262 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 22:20:10.779371 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.199827 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 22:20:10.779371 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.199853 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-114.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 22:20:10.779371 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.201012 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.201022 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.201030 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.201465 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.202550 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.206198 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.206323 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.206351 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.206360 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.206376 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.206386 2578 kubelet.go:398] "Adding apiserver pod source" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.206396 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.206765 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.206789 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.212285 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.214159 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.214187 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215200 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215213 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215220 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215225 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215234 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215239 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215251 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215260 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215268 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215274 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215283 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215288 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215901 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.215908 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.218666 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.218680 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.218691 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-114.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.219425 2578 server.go:1268] "Started kubelet" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.219481 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.219585 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.219675 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.220716 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.221208 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 06 22:20:10.779675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.224974 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kf8xq" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.226453 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.226823 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.227543 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.228879 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.228892 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.227579 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.228984 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.228989 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.230179 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.231333 2578 factory.go:55] Registering systemd factory Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.231348 2578 factory.go:223] Registration of the systemd container factory successfully Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.231601 2578 factory.go:153] Registering CRI-O factory Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.231617 2578 factory.go:223] Registration of the crio container factory successfully Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.231672 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.231697 2578 factory.go:103] Registering Raw factory Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.231711 2578 manager.go:1196] Started watching for new ooms in manager Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.232035 2578 manager.go:349] Starting recovery of all containers Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.232248 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kf8xq" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.234761 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-114.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.234882 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.234946 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-114.ec2.internal.18bfd36577f03b84 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-114.ec2.internal,UID:ip-10-0-142-114.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-114.ec2.internal,},FirstTimestamp:2026-07-06 22:20:10.219395972 +0000 UTC m=+0.321123625,LastTimestamp:2026-07-06 22:20:10.219395972 +0000 UTC m=+0.321123625,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-114.ec2.internal,}" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.242057 2578 manager.go:354] Recovery completed Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.245964 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.247795 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.247829 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.247844 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.248457 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.248468 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.248468 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.248543 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.249141 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:20:10.780751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.249164 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.249191 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.250745 2578 policy_none.go:49] "None policy: Start" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.250757 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.250768 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.252822 2578 policy_none.go:47] "Start" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.280218 2578 manager.go:337] "Starting Device Plugin manager" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.280426 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.280439 2578 server.go:87] "Starting device plugin registration server" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.280715 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.280728 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.281316 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.281399 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.281407 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.282720 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.283021 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.367484 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.372272 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.372308 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.372334 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.372370 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.375139 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.383423 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.384271 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.384294 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.384305 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.384325 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.400250 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.400280 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-114.ec2.internal\": node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.418245 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.473268 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal"] Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.473351 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.474840 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.474865 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.474874 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.477638 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.477696 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.477724 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:20:10.781678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.478472 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.478491 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.478501 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.478954 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.478984 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.478996 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.480681 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.480703 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.481313 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.481339 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.481350 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.505943 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-114.ec2.internal\" not found" node="ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.509630 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-114.ec2.internal\" not found" node="ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.518597 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.530344 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/57a5e8dc5f4b0683c61c1d81a1152807-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal\" (UID: \"57a5e8dc5f4b0683c61c1d81a1152807\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.530367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57a5e8dc5f4b0683c61c1d81a1152807-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal\" (UID: \"57a5e8dc5f4b0683c61c1d81a1152807\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.530387 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b2e27a26a564387fecc3ee91d617486-config\") pod \"kube-apiserver-proxy-ip-10-0-142-114.ec2.internal\" (UID: \"7b2e27a26a564387fecc3ee91d617486\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.618789 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.631199 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/57a5e8dc5f4b0683c61c1d81a1152807-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal\" (UID: \"57a5e8dc5f4b0683c61c1d81a1152807\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.631227 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57a5e8dc5f4b0683c61c1d81a1152807-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal\" (UID: \"57a5e8dc5f4b0683c61c1d81a1152807\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.631246 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b2e27a26a564387fecc3ee91d617486-config\") pod \"kube-apiserver-proxy-ip-10-0-142-114.ec2.internal\" (UID: \"7b2e27a26a564387fecc3ee91d617486\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.782691 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.631296 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/57a5e8dc5f4b0683c61c1d81a1152807-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal\" (UID: \"57a5e8dc5f4b0683c61c1d81a1152807\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.783353 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.631308 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/57a5e8dc5f4b0683c61c1d81a1152807-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal\" (UID: \"57a5e8dc5f4b0683c61c1d81a1152807\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.783353 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.631351 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7b2e27a26a564387fecc3ee91d617486-config\") pod \"kube-apiserver-proxy-ip-10-0-142-114.ec2.internal\" (UID: \"7b2e27a26a564387fecc3ee91d617486\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.783353 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.719654 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:10.991820 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.808168 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.991820 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:10.811858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal" Jul 06 22:20:10.991820 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.823023 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:10.991820 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:10.923622 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:11.183133 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:11.024162 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:11.183133 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:11.124863 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:11.183133 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.171376 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 22:20:11.183133 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.171514 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 22:20:11.342893 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:11.225159 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:11.342893 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.237226 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 22:15:10 +0000 UTC" deadline="2028-03-07 11:58:42.229090478 +0000 UTC" Jul 06 22:20:11.342893 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.237251 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14629h38m30.99184275s" Jul 06 22:20:11.342893 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:11.325920 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:11.535411 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:11.398167 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2e27a26a564387fecc3ee91d617486.slice/crio-185a82e0905b7b83d190582cc313ff3dc19767e213a7fa3c1567080e4568529a WatchSource:0}: Error finding container 185a82e0905b7b83d190582cc313ff3dc19767e213a7fa3c1567080e4568529a: Status 404 returned error can't find the container with id 185a82e0905b7b83d190582cc313ff3dc19767e213a7fa3c1567080e4568529a Jul 06 22:20:11.535411 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.410276 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:20:11.535411 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:11.416918 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a5e8dc5f4b0683c61c1d81a1152807.slice/crio-43c5d8ce43ed72e20441ea2f6115e9cccfc507d5bd9ca742bdb688a0b6d648dc WatchSource:0}: Error finding container 43c5d8ce43ed72e20441ea2f6115e9cccfc507d5bd9ca742bdb688a0b6d648dc: Status 404 returned error can't find the container with id 43c5d8ce43ed72e20441ea2f6115e9cccfc507d5bd9ca742bdb688a0b6d648dc Jul 06 22:20:11.535411 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:11.426897 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:11.535411 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.484503 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:20:11.535411 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:11.527303 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-114.ec2.internal\" not found" Jul 06 22:20:11.766620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.579027 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:20:11.766620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.628818 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" Jul 06 22:20:11.766620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.640465 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 22:20:11.766620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.641828 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal" Jul 06 22:20:11.766620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.650976 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 22:20:11.766620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:11.680394 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:20:12.207738 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.207427 2578 apiserver.go:52] "Watching apiserver" Jul 06 22:20:12.214819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.214012 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 22:20:12.214819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.214569 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-gzgkl","openshift-multus/network-metrics-daemon-xl9br","kube-system/global-pull-secret-syncer-wmpgs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj","openshift-image-registry/node-ca-w9g5x","openshift-network-diagnostics/network-check-target-8hnkk","openshift-network-operator/iptables-alerter-q77mn","openshift-ovn-kubernetes/ovnkube-node-ptrf4","kube-system/konnectivity-agent-6n28g","kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rbbsg","openshift-dns/node-resolver-k7phn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal","openshift-multus/multus-additional-cni-plugins-z29kj"] Jul 06 22:20:12.217189 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.217162 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.219523 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.219497 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qtp99\"" Jul 06 22:20:12.219968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.219819 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 22:20:12.219968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.219861 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:20:12.219968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.219822 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 22:20:12.220206 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.220077 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.223180 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.223157 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.223334 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.223299 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 22:20:12.223455 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.223427 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:12.224316 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.223749 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mpxbz\"" Jul 06 22:20:12.224316 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.224001 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:20:12.228200 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.227350 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.228200 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.227363 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 22:20:12.229385 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.229364 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.230415 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.230061 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rgv2c\"" Jul 06 22:20:12.230415 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.230206 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 22:20:12.230415 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.230275 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 22:20:12.230596 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.230438 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 22:20:12.231479 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.231323 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 22:20:12.231759 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.231648 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:12.231759 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.231722 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:12.232133 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.232097 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 22:20:12.232422 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.232107 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-56btr\"" Jul 06 22:20:12.232611 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.232590 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 22:20:12.234097 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.233825 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:12.236087 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.235912 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dkxxp\"" Jul 06 22:20:12.236087 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.235912 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 22:20:12.236087 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.235968 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 22:20:12.236265 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.236210 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.237079 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.236806 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 22:20:12.238388 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.238366 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 22:20:12.238532 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.238515 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 22:20:12.238699 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.238643 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 22:20:12.238699 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.238520 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t7m6v\"" Jul 06 22:20:12.238800 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.238371 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 22:20:12.238896 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.238879 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:12.238977 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.238954 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:12.239259 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.239243 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 22:20:12.240198 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-sysconfig\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240277 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-sys\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240277 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240224 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-device-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.240277 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-modprobe-d\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240277 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240257 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-kubernetes\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240284 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-systemd\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13f5b744-1211-4dab-bf92-1b298db35e5a-tmp\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240317 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-registration-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.240486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8kpq\" (UniqueName: \"kubernetes.io/projected/13f5b744-1211-4dab-bf92-1b298db35e5a-kube-api-access-x8kpq\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240372 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.240486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.240486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.240486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240468 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-host\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240511 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15174860-7d96-48c1-8616-5ddabcff08e8-host-slash\") pod \"iptables-alerter-q77mn\" (UID: \"15174860-7d96-48c1-8616-5ddabcff08e8\") " pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240542 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn9rw\" (UniqueName: \"kubernetes.io/projected/15174860-7d96-48c1-8616-5ddabcff08e8-kube-api-access-pn9rw\") pod \"iptables-alerter-q77mn\" (UID: \"15174860-7d96-48c1-8616-5ddabcff08e8\") " pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-run\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-lib-modules\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240625 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-sys-fs\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240653 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5517402c-0226-4bbb-8d38-0200eb62df31-serviceca\") pod \"node-ca-w9g5x\" (UID: \"5517402c-0226-4bbb-8d38-0200eb62df31\") " pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15174860-7d96-48c1-8616-5ddabcff08e8-iptables-alerter-script\") pod \"iptables-alerter-q77mn\" (UID: \"15174860-7d96-48c1-8616-5ddabcff08e8\") " pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240712 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-sysctl-d\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-tuned\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d1befd3e-57bc-4d94-9563-ef9479a3d10c-kubelet-config\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240789 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5517402c-0226-4bbb-8d38-0200eb62df31-host\") pod \"node-ca-w9g5x\" (UID: \"5517402c-0226-4bbb-8d38-0200eb62df31\") " pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240809 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzk2n\" (UniqueName: \"kubernetes.io/projected/5517402c-0226-4bbb-8d38-0200eb62df31-kube-api-access-hzk2n\") pod \"node-ca-w9g5x\" (UID: \"5517402c-0226-4bbb-8d38-0200eb62df31\") " pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.240844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6xfh\" (UniqueName: \"kubernetes.io/projected/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-kube-api-access-v6xfh\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.241467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240872 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-sysctl-conf\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.241467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240880 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.241467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240910 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-var-lib-kubelet\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.241467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240937 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d1befd3e-57bc-4d94-9563-ef9479a3d10c-dbus\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.241467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.240963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-socket-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.243351 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.242694 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 22:20:12.243351 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.242763 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 22:20:12.244430 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.243858 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.244430 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.243916 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 22:20:12.244430 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.243988 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 22:20:12.244430 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.244156 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 22:20:12.244430 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.244183 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 22:20:12.244713 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.244508 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jhwq7\"" Jul 06 22:20:12.245795 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.245776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.245881 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.245820 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 22:20:12.246205 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.246042 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fhq8g\"" Jul 06 22:20:12.248590 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.247732 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 22:20:12.248590 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.248157 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 22:20:12.248590 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.248472 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xk5qq\"" Jul 06 22:20:12.266126 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.266094 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8bqwf" Jul 06 22:20:12.274430 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.274408 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8bqwf" Jul 06 22:20:12.329126 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.329022 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 22:20:12.341945 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.341912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.341945 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.341949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.342186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.341966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-cni-bin\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.342186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.341980 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-cni-dir\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.342186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.341995 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-socket-dir-parent\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.342186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342010 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-daemon-config\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.342186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-var-lib-openvswitch\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.342186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de18a133-ab7e-4483-a93a-bbdee1fbc449-ovnkube-config\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.342186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15174860-7d96-48c1-8616-5ddabcff08e8-host-slash\") pod \"iptables-alerter-q77mn\" (UID: \"15174860-7d96-48c1-8616-5ddabcff08e8\") " pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.342186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342132 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-etc-selinux\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.342186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342142 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-cni-binary-copy\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342192 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-var-lib-cni-bin\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342215 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5517402c-0226-4bbb-8d38-0200eb62df31-serviceca\") pod \"node-ca-w9g5x\" (UID: \"5517402c-0226-4bbb-8d38-0200eb62df31\") " pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342191 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15174860-7d96-48c1-8616-5ddabcff08e8-host-slash\") pod \"iptables-alerter-q77mn\" (UID: \"15174860-7d96-48c1-8616-5ddabcff08e8\") " pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342236 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15174860-7d96-48c1-8616-5ddabcff08e8-iptables-alerter-script\") pod \"iptables-alerter-q77mn\" (UID: \"15174860-7d96-48c1-8616-5ddabcff08e8\") " pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342260 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-tuned\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342288 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d1befd3e-57bc-4d94-9563-ef9479a3d10c-kubelet-config\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342311 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-slash\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342346 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-run-ovn\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342373 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzk2n\" (UniqueName: \"kubernetes.io/projected/5517402c-0226-4bbb-8d38-0200eb62df31-kube-api-access-hzk2n\") pod \"node-ca-w9g5x\" (UID: \"5517402c-0226-4bbb-8d38-0200eb62df31\") " pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xfh\" (UniqueName: \"kubernetes.io/projected/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-kube-api-access-v6xfh\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9r8x\" (UniqueName: \"kubernetes.io/projected/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-kube-api-access-g9r8x\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342449 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-var-lib-kubelet\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342472 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-sysctl-conf\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.342604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-socket-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342522 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-os-release\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342545 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de18a133-ab7e-4483-a93a-bbdee1fbc449-ovnkube-script-lib\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342568 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/281a0cf8-525d-4171-b0f3-8020a15bfd48-hosts-file\") pod \"node-resolver-k7phn\" (UID: \"281a0cf8-525d-4171-b0f3-8020a15bfd48\") " pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-cnibin\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342615 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-systemd-units\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-sysconfig\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342667 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-sys\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342691 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-device-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342716 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-cni-binary-copy\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342741 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5517402c-0226-4bbb-8d38-0200eb62df31-serviceca\") pod \"node-ca-w9g5x\" (UID: \"5517402c-0226-4bbb-8d38-0200eb62df31\") " pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342788 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-systemd\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.343801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13f5b744-1211-4dab-bf92-1b298db35e5a-tmp\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342843 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-registration-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342843 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15174860-7d96-48c1-8616-5ddabcff08e8-iptables-alerter-script\") pod \"iptables-alerter-q77mn\" (UID: \"15174860-7d96-48c1-8616-5ddabcff08e8\") " pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342866 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-cnibin\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-kubelet\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.342999 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-sysconfig\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343043 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343064 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-sys\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343124 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-device-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343178 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d1befd3e-57bc-4d94-9563-ef9479a3d10c-kubelet-config\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343249 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-systemd\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343352 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-run-ovn-kubernetes\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343385 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-run-k8s-cni-cncf-io\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-hostroot\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.344649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8kpq\" (UniqueName: \"kubernetes.io/projected/13f5b744-1211-4dab-bf92-1b298db35e5a-kube-api-access-x8kpq\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-run-systemd\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de18a133-ab7e-4483-a93a-bbdee1fbc449-env-overrides\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-var-lib-cni-multus\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343528 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-etc-kubernetes\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.343695 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343701 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-socket-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343700 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-registration-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343733 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.343755 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret podName:d1befd3e-57bc-4d94-9563-ef9479a3d10c nodeName:}" failed. No retries permitted until 2026-07-06 22:20:12.843735277 +0000 UTC m=+2.945462925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret") pod "global-pull-secret-syncer-wmpgs" (UID: "d1befd3e-57bc-4d94-9563-ef9479a3d10c") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343773 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343828 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-sysctl-conf\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.345302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343853 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-conf-dir\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-host\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343896 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-host\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343918 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn9rw\" (UniqueName: \"kubernetes.io/projected/15174860-7d96-48c1-8616-5ddabcff08e8-kube-api-access-pn9rw\") pod \"iptables-alerter-q77mn\" (UID: \"15174860-7d96-48c1-8616-5ddabcff08e8\") " pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.343957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-run\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-lib-modules\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344084 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-sys-fs\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344107 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-etc-openvswitch\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344159 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344162 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-run\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-sysctl-d\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-system-cni-dir\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344229 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-sys-fs\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344249 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-lib-modules\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.345961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344263 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0-agent-certs\") pod \"konnectivity-agent-6n28g\" (UID: \"7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0\") " pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-system-cni-dir\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344296 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-sysctl-d\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344320 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5517402c-0226-4bbb-8d38-0200eb62df31-host\") pod \"node-ca-w9g5x\" (UID: \"5517402c-0226-4bbb-8d38-0200eb62df31\") " pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-run-netns\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-run-openvswitch\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344409 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-node-log\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344430 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-cni-netd\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344384 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5517402c-0226-4bbb-8d38-0200eb62df31-host\") pod \"node-ca-w9g5x\" (UID: \"5517402c-0226-4bbb-8d38-0200eb62df31\") " pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344454 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpzt8\" (UniqueName: \"kubernetes.io/projected/de18a133-ab7e-4483-a93a-bbdee1fbc449-kube-api-access-mpzt8\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0-konnectivity-ca\") pod \"konnectivity-agent-6n28g\" (UID: \"7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0\") " pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-var-lib-kubelet\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d1befd3e-57bc-4d94-9563-ef9479a3d10c-dbus\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344570 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-var-lib-kubelet\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.346647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344576 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344601 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-log-socket\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344621 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/281a0cf8-525d-4171-b0f3-8020a15bfd48-tmp-dir\") pod \"node-resolver-k7phn\" (UID: \"281a0cf8-525d-4171-b0f3-8020a15bfd48\") " pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-os-release\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfz9r\" (UniqueName: \"kubernetes.io/projected/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-kube-api-access-rfz9r\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344669 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d1befd3e-57bc-4d94-9563-ef9479a3d10c-dbus\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344690 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-modprobe-d\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344713 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-kubernetes\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de18a133-ab7e-4483-a93a-bbdee1fbc449-ovn-node-metrics-cert\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgkgl\" (UniqueName: \"kubernetes.io/projected/281a0cf8-525d-4171-b0f3-8020a15bfd48-kube-api-access-tgkgl\") pod \"node-resolver-k7phn\" (UID: \"281a0cf8-525d-4171-b0f3-8020a15bfd48\") " pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344783 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-run-netns\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344790 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-modprobe-d\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-run-multus-certs\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344831 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dchbp\" (UniqueName: \"kubernetes.io/projected/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-kube-api-access-dchbp\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.347293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.344846 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-kubernetes\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.347926 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.346767 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/13f5b744-1211-4dab-bf92-1b298db35e5a-etc-tuned\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.347926 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.347048 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13f5b744-1211-4dab-bf92-1b298db35e5a-tmp\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.359241 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.359177 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzk2n\" (UniqueName: \"kubernetes.io/projected/5517402c-0226-4bbb-8d38-0200eb62df31-kube-api-access-hzk2n\") pod \"node-ca-w9g5x\" (UID: \"5517402c-0226-4bbb-8d38-0200eb62df31\") " pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.359661 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.359642 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8kpq\" (UniqueName: \"kubernetes.io/projected/13f5b744-1211-4dab-bf92-1b298db35e5a-kube-api-access-x8kpq\") pod \"tuned-rbbsg\" (UID: \"13f5b744-1211-4dab-bf92-1b298db35e5a\") " pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.359764 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.359748 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6xfh\" (UniqueName: \"kubernetes.io/projected/dd5fd205-1e42-4ce6-8f6e-ca04f80384aa-kube-api-access-v6xfh\") pod \"aws-ebs-csi-driver-node-xmzgj\" (UID: \"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.360376 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.360349 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn9rw\" (UniqueName: \"kubernetes.io/projected/15174860-7d96-48c1-8616-5ddabcff08e8-kube-api-access-pn9rw\") pod \"iptables-alerter-q77mn\" (UID: \"15174860-7d96-48c1-8616-5ddabcff08e8\") " pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.379487 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.379402 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" event={"ID":"57a5e8dc5f4b0683c61c1d81a1152807","Type":"ContainerStarted","Data":"43c5d8ce43ed72e20441ea2f6115e9cccfc507d5bd9ca742bdb688a0b6d648dc"} Jul 06 22:20:12.380836 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.380779 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal" event={"ID":"7b2e27a26a564387fecc3ee91d617486","Type":"ContainerStarted","Data":"185a82e0905b7b83d190582cc313ff3dc19767e213a7fa3c1567080e4568529a"} Jul 06 22:20:12.445249 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445213 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-system-cni-dir\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.445392 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-run-netns\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445392 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-run-openvswitch\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445392 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-node-log\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445392 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-cni-netd\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445392 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpzt8\" (UniqueName: \"kubernetes.io/projected/de18a133-ab7e-4483-a93a-bbdee1fbc449-kube-api-access-mpzt8\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445392 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445349 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-run-openvswitch\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445392 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445351 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-system-cni-dir\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.445392 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445346 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-run-netns\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445393 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-node-log\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0-konnectivity-ca\") pod \"konnectivity-agent-6n28g\" (UID: \"7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0\") " pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445484 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-cni-netd\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445534 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-log-socket\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/281a0cf8-525d-4171-b0f3-8020a15bfd48-tmp-dir\") pod \"node-resolver-k7phn\" (UID: \"281a0cf8-525d-4171-b0f3-8020a15bfd48\") " pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-os-release\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445611 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfz9r\" (UniqueName: \"kubernetes.io/projected/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-kube-api-access-rfz9r\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445618 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-log-socket\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de18a133-ab7e-4483-a93a-bbdee1fbc449-ovn-node-metrics-cert\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445667 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgkgl\" (UniqueName: \"kubernetes.io/projected/281a0cf8-525d-4171-b0f3-8020a15bfd48-kube-api-access-tgkgl\") pod \"node-resolver-k7phn\" (UID: \"281a0cf8-525d-4171-b0f3-8020a15bfd48\") " pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445686 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-os-release\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445690 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-run-netns\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445731 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-run-multus-certs\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.445785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445760 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dchbp\" (UniqueName: \"kubernetes.io/projected/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-kube-api-access-dchbp\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445786 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445842 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-cni-bin\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445853 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/281a0cf8-525d-4171-b0f3-8020a15bfd48-tmp-dir\") pod \"node-resolver-k7phn\" (UID: \"281a0cf8-525d-4171-b0f3-8020a15bfd48\") " pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-cni-dir\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445761 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-run-netns\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-socket-dir-parent\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-daemon-config\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445920 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-run-multus-certs\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-var-lib-openvswitch\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.445988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de18a133-ab7e-4483-a93a-bbdee1fbc449-ovnkube-config\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-cni-binary-copy\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446042 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-var-lib-cni-bin\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446062 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0-konnectivity-ca\") pod \"konnectivity-agent-6n28g\" (UID: \"7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0\") " pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446070 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-slash\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.446518 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-run-ovn\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9r8x\" (UniqueName: \"kubernetes.io/projected/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-kube-api-access-g9r8x\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446172 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-cni-bin\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-var-lib-kubelet\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-os-release\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446248 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de18a133-ab7e-4483-a93a-bbdee1fbc449-ovnkube-script-lib\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/281a0cf8-525d-4171-b0f3-8020a15bfd48-hosts-file\") pod \"node-resolver-k7phn\" (UID: \"281a0cf8-525d-4171-b0f3-8020a15bfd48\") " pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446321 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-run-ovn\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446344 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-var-lib-openvswitch\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446345 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-var-lib-kubelet\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446355 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-os-release\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446344 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-var-lib-cni-bin\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446376 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-cnibin\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446404 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-slash\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446428 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-systemd-units\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446452 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-socket-dir-parent\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-cni-binary-copy\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446482 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-cnibin\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446491 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446493 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/281a0cf8-525d-4171-b0f3-8020a15bfd48-hosts-file\") pod \"node-resolver-k7phn\" (UID: \"281a0cf8-525d-4171-b0f3-8020a15bfd48\") " pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-cnibin\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446576 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-systemd-units\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-kubelet\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.446602 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446604 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-run-ovn-kubernetes\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-run-k8s-cni-cncf-io\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.447912 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.446653 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs podName:e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d nodeName:}" failed. No retries permitted until 2026-07-06 22:20:12.946636629 +0000 UTC m=+3.048364271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs") pod "network-metrics-daemon-xl9br" (UID: "e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446671 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-run-k8s-cni-cncf-io\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-hostroot\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-run-systemd\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446713 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-cnibin\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446741 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446746 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-kubelet\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446773 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-run-ovn-kubernetes\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446794 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-hostroot\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446840 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-run-systemd\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.446904 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de18a133-ab7e-4483-a93a-bbdee1fbc449-ovnkube-config\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447020 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-cni-dir\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447060 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-cni-binary-copy\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de18a133-ab7e-4483-a93a-bbdee1fbc449-env-overrides\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-var-lib-cni-multus\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.448562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447281 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de18a133-ab7e-4483-a93a-bbdee1fbc449-env-overrides\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-etc-kubernetes\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447334 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-host-var-lib-cni-multus\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-conf-dir\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447361 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de18a133-ab7e-4483-a93a-bbdee1fbc449-ovnkube-script-lib\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447377 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-etc-kubernetes\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-etc-openvswitch\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447408 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447417 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-conf-dir\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-system-cni-dir\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447450 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0-agent-certs\") pod \"konnectivity-agent-6n28g\" (UID: \"7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0\") " pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de18a133-ab7e-4483-a93a-bbdee1fbc449-etc-openvswitch\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.449170 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447472 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-system-cni-dir\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.449649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.447926 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-multus-daemon-config\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.449649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.448050 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-cni-binary-copy\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.449649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.448861 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de18a133-ab7e-4483-a93a-bbdee1fbc449-ovn-node-metrics-cert\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.450287 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.450226 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0-agent-certs\") pod \"konnectivity-agent-6n28g\" (UID: \"7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0\") " pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:12.453975 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.453955 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:20:12.454154 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.454131 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:20:12.454154 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.454156 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zk4lv for pod openshift-network-diagnostics/network-check-target-8hnkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:12.454333 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.454279 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv podName:edfcc999-de23-48fe-8d7d-72fb9836aa2b nodeName:}" failed. No retries permitted until 2026-07-06 22:20:12.954261768 +0000 UTC m=+3.055989421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zk4lv" (UniqueName: "kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv") pod "network-check-target-8hnkk" (UID: "edfcc999-de23-48fe-8d7d-72fb9836aa2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:12.455765 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.455728 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9r8x\" (UniqueName: \"kubernetes.io/projected/96ce3e1e-19db-4ebc-8d09-8c6a962cefd0-kube-api-access-g9r8x\") pod \"multus-additional-cni-plugins-z29kj\" (UID: \"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0\") " pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.455867 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.455822 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfz9r\" (UniqueName: \"kubernetes.io/projected/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-kube-api-access-rfz9r\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:12.455867 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.455842 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dchbp\" (UniqueName: \"kubernetes.io/projected/690ba9a7-5ee6-4aeb-93ea-5e087ae83a55-kube-api-access-dchbp\") pod \"multus-gzgkl\" (UID: \"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55\") " pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.456187 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.456164 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgkgl\" (UniqueName: \"kubernetes.io/projected/281a0cf8-525d-4171-b0f3-8020a15bfd48-kube-api-access-tgkgl\") pod \"node-resolver-k7phn\" (UID: \"281a0cf8-525d-4171-b0f3-8020a15bfd48\") " pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.456291 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.456271 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpzt8\" (UniqueName: \"kubernetes.io/projected/de18a133-ab7e-4483-a93a-bbdee1fbc449-kube-api-access-mpzt8\") pod \"ovnkube-node-ptrf4\" (UID: \"de18a133-ab7e-4483-a93a-bbdee1fbc449\") " pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.535247 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.535043 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q77mn" Jul 06 22:20:12.543046 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.543021 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" Jul 06 22:20:12.553963 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.553940 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" Jul 06 22:20:12.559767 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.559742 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w9g5x" Jul 06 22:20:12.567612 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.567590 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:12.575227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.575204 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z29kj" Jul 06 22:20:12.586001 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.585978 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:12.594871 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.594848 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gzgkl" Jul 06 22:20:12.600543 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.600516 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k7phn" Jul 06 22:20:12.729261 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.729224 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:20:12.849783 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.849754 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:12.849958 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.849918 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:12.850019 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.849974 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret podName:d1befd3e-57bc-4d94-9563-ef9479a3d10c nodeName:}" failed. No retries permitted until 2026-07-06 22:20:13.849960457 +0000 UTC m=+3.951688097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret") pod "global-pull-secret-syncer-wmpgs" (UID: "d1befd3e-57bc-4d94-9563-ef9479a3d10c") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:12.950650 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:12.950559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:12.950815 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.950694 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:12.950815 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:12.950758 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs podName:e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d nodeName:}" failed. No retries permitted until 2026-07-06 22:20:13.950740394 +0000 UTC m=+4.052468041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs") pod "network-metrics-daemon-xl9br" (UID: "e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:13.051814 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.051782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:13.051962 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:13.051912 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:20:13.051962 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:13.051937 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:20:13.051962 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:13.051947 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zk4lv for pod openshift-network-diagnostics/network-check-target-8hnkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:13.052061 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:13.052011 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv podName:edfcc999-de23-48fe-8d7d-72fb9836aa2b nodeName:}" failed. No retries permitted until 2026-07-06 22:20:14.051991031 +0000 UTC m=+4.153718689 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk4lv" (UniqueName: "kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv") pod "network-check-target-8hnkk" (UID: "edfcc999-de23-48fe-8d7d-72fb9836aa2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:13.277895 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.277766 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:15:12 +0000 UTC" deadline="2028-02-23 17:13:53.075263895 +0000 UTC" Jul 06 22:20:13.277895 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.277844 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14322h53m39.797424009s" Jul 06 22:20:13.353345 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:13.353313 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690ba9a7_5ee6_4aeb_93ea_5e087ae83a55.slice/crio-fc1c32bb2ce09a36c87d32267aa5e974186f8329986e9ae82160101d40199dd4 WatchSource:0}: Error finding container fc1c32bb2ce09a36c87d32267aa5e974186f8329986e9ae82160101d40199dd4: Status 404 returned error can't find the container with id fc1c32bb2ce09a36c87d32267aa5e974186f8329986e9ae82160101d40199dd4 Jul 06 22:20:13.354980 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:13.354855 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ce3e1e_19db_4ebc_8d09_8c6a962cefd0.slice/crio-697e06873154b670d8070b42d777ac5102451e00f48a8af09077dc17adc781b5 WatchSource:0}: Error finding container 697e06873154b670d8070b42d777ac5102451e00f48a8af09077dc17adc781b5: Status 404 returned error can't find the container with id 697e06873154b670d8070b42d777ac5102451e00f48a8af09077dc17adc781b5 Jul 06 22:20:13.358836 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:13.358811 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5517402c_0226_4bbb_8d38_0200eb62df31.slice/crio-eae146448e4423ac14510aaeeb65166a6bcbe8246c2c51143ec0f1760428468a WatchSource:0}: Error finding container eae146448e4423ac14510aaeeb65166a6bcbe8246c2c51143ec0f1760428468a: Status 404 returned error can't find the container with id eae146448e4423ac14510aaeeb65166a6bcbe8246c2c51143ec0f1760428468a Jul 06 22:20:13.361995 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:13.361972 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde18a133_ab7e_4483_a93a_bbdee1fbc449.slice/crio-e639313dc7f12b9ec64986991beb91f89ba83301987020cab6cf6638eb71f487 WatchSource:0}: Error finding container e639313dc7f12b9ec64986991beb91f89ba83301987020cab6cf6638eb71f487: Status 404 returned error can't find the container with id e639313dc7f12b9ec64986991beb91f89ba83301987020cab6cf6638eb71f487 Jul 06 22:20:13.363015 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:13.362991 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f5b744_1211_4dab_bf92_1b298db35e5a.slice/crio-4f015f8f1f05c551639d34becc8508eed31a74471236288c0e84d2762f893bfb WatchSource:0}: Error finding container 4f015f8f1f05c551639d34becc8508eed31a74471236288c0e84d2762f893bfb: Status 404 returned error can't find the container with id 4f015f8f1f05c551639d34becc8508eed31a74471236288c0e84d2762f893bfb Jul 06 22:20:13.364622 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:13.364559 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ddb25aa_2738_45d5_9b3a_5e4ed8e2eae0.slice/crio-77c6bd5fe667095c970b21cef74d923093bab9bc2ea683ea39095e0c8f4b01da WatchSource:0}: Error finding container 77c6bd5fe667095c970b21cef74d923093bab9bc2ea683ea39095e0c8f4b01da: Status 404 returned error can't find the container with id 77c6bd5fe667095c970b21cef74d923093bab9bc2ea683ea39095e0c8f4b01da Jul 06 22:20:13.366412 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:13.366393 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15174860_7d96_48c1_8616_5ddabcff08e8.slice/crio-92201fe24c45094ee0957b5295c1d73d22717d62cf8f2df44176b1d917f0cef1 WatchSource:0}: Error finding container 92201fe24c45094ee0957b5295c1d73d22717d62cf8f2df44176b1d917f0cef1: Status 404 returned error can't find the container with id 92201fe24c45094ee0957b5295c1d73d22717d62cf8f2df44176b1d917f0cef1 Jul 06 22:20:13.367742 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:13.367717 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5fd205_1e42_4ce6_8f6e_ca04f80384aa.slice/crio-fa6430fdc548f38fce8a39e734aaf412a49cd4c4cd4d8b0f9c547de0e0e7581b WatchSource:0}: Error finding container fa6430fdc548f38fce8a39e734aaf412a49cd4c4cd4d8b0f9c547de0e0e7581b: Status 404 returned error can't find the container with id fa6430fdc548f38fce8a39e734aaf412a49cd4c4cd4d8b0f9c547de0e0e7581b Jul 06 22:20:13.383217 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.383185 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k7phn" event={"ID":"281a0cf8-525d-4171-b0f3-8020a15bfd48","Type":"ContainerStarted","Data":"a9fbbedc8d6db7b1a9370de582749c3d5f647473b4e4b1df89d2254c0e8d39d9"} Jul 06 22:20:13.384338 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.384317 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" event={"ID":"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa","Type":"ContainerStarted","Data":"fa6430fdc548f38fce8a39e734aaf412a49cd4c4cd4d8b0f9c547de0e0e7581b"} Jul 06 22:20:13.385479 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.385453 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" event={"ID":"de18a133-ab7e-4483-a93a-bbdee1fbc449","Type":"ContainerStarted","Data":"e639313dc7f12b9ec64986991beb91f89ba83301987020cab6cf6638eb71f487"} Jul 06 22:20:13.386569 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.386537 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z29kj" event={"ID":"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0","Type":"ContainerStarted","Data":"697e06873154b670d8070b42d777ac5102451e00f48a8af09077dc17adc781b5"} Jul 06 22:20:13.387785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.387761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gzgkl" event={"ID":"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55","Type":"ContainerStarted","Data":"fc1c32bb2ce09a36c87d32267aa5e974186f8329986e9ae82160101d40199dd4"} Jul 06 22:20:13.388897 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.388879 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q77mn" event={"ID":"15174860-7d96-48c1-8616-5ddabcff08e8","Type":"ContainerStarted","Data":"92201fe24c45094ee0957b5295c1d73d22717d62cf8f2df44176b1d917f0cef1"} Jul 06 22:20:13.390002 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.389973 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6n28g" event={"ID":"7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0","Type":"ContainerStarted","Data":"77c6bd5fe667095c970b21cef74d923093bab9bc2ea683ea39095e0c8f4b01da"} Jul 06 22:20:13.391104 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.391085 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" event={"ID":"13f5b744-1211-4dab-bf92-1b298db35e5a","Type":"ContainerStarted","Data":"4f015f8f1f05c551639d34becc8508eed31a74471236288c0e84d2762f893bfb"} Jul 06 22:20:13.392742 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.392717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w9g5x" event={"ID":"5517402c-0226-4bbb-8d38-0200eb62df31","Type":"ContainerStarted","Data":"eae146448e4423ac14510aaeeb65166a6bcbe8246c2c51143ec0f1760428468a"} Jul 06 22:20:13.857926 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.857663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:13.858128 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:13.857997 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:13.858128 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:13.858074 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret podName:d1befd3e-57bc-4d94-9563-ef9479a3d10c nodeName:}" failed. No retries permitted until 2026-07-06 22:20:15.858055106 +0000 UTC m=+5.959782763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret") pod "global-pull-secret-syncer-wmpgs" (UID: "d1befd3e-57bc-4d94-9563-ef9479a3d10c") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:13.959016 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:13.958977 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:13.959222 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:13.959132 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:13.959222 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:13.959211 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs podName:e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d nodeName:}" failed. No retries permitted until 2026-07-06 22:20:15.959179498 +0000 UTC m=+6.060907146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs") pod "network-metrics-daemon-xl9br" (UID: "e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:14.060445 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.060268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:14.060445 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:14.060441 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:20:14.060683 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:14.060465 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:20:14.060683 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:14.060477 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zk4lv for pod openshift-network-diagnostics/network-check-target-8hnkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:14.060683 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:14.060540 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv podName:edfcc999-de23-48fe-8d7d-72fb9836aa2b nodeName:}" failed. No retries permitted until 2026-07-06 22:20:16.06052121 +0000 UTC m=+6.162248863 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk4lv" (UniqueName: "kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv") pod "network-check-target-8hnkk" (UID: "edfcc999-de23-48fe-8d7d-72fb9836aa2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:14.279254 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.278858 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:15:12 +0000 UTC" deadline="2028-01-13 14:09:45.871943377 +0000 UTC" Jul 06 22:20:14.279254 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.278896 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13335h49m31.593051666s" Jul 06 22:20:14.373281 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.373229 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:14.373281 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.373250 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:14.373909 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:14.373360 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:14.373909 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.373428 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:14.373909 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:14.373545 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:14.373909 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:14.373623 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:14.404290 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.404242 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal" event={"ID":"7b2e27a26a564387fecc3ee91d617486","Type":"ContainerStarted","Data":"1a21de41b159fe351e1e7af0b50adc8b9e36187e3e0913954e34b1e0ac56a824"} Jul 06 22:20:14.409940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.409908 2578 generic.go:356] "Generic (PLEG): container finished" podID="57a5e8dc5f4b0683c61c1d81a1152807" containerID="c383b70fd54677662a48a234749cdf9d6a8c1e3fb2a419b918399942b41f3f44" exitCode=0 Jul 06 22:20:14.410098 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.409960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" event={"ID":"57a5e8dc5f4b0683c61c1d81a1152807","Type":"ContainerDied","Data":"c383b70fd54677662a48a234749cdf9d6a8c1e3fb2a419b918399942b41f3f44"} Jul 06 22:20:14.421785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:14.421650 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-114.ec2.internal" podStartSLOduration=3.42163213 podStartE2EDuration="3.42163213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:14.420106808 +0000 UTC m=+4.521834471" watchObservedRunningTime="2026-07-06 22:20:14.42163213 +0000 UTC m=+4.523359792" Jul 06 22:20:15.435654 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:15.435618 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" event={"ID":"57a5e8dc5f4b0683c61c1d81a1152807","Type":"ContainerStarted","Data":"ae1fe207ca70deebcae480c3f9d35a5438df1974188bcd5a4960781383e5957c"} Jul 06 22:20:15.452560 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:15.452501 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-114.ec2.internal" podStartSLOduration=4.452479169 podStartE2EDuration="4.452479169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:15.452357432 +0000 UTC m=+5.554085094" watchObservedRunningTime="2026-07-06 22:20:15.452479169 +0000 UTC m=+5.554206830" Jul 06 22:20:15.874701 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:15.874658 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:15.874890 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:15.874846 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:15.874952 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:15.874911 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret podName:d1befd3e-57bc-4d94-9563-ef9479a3d10c nodeName:}" failed. No retries permitted until 2026-07-06 22:20:19.874892856 +0000 UTC m=+9.976620509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret") pod "global-pull-secret-syncer-wmpgs" (UID: "d1befd3e-57bc-4d94-9563-ef9479a3d10c") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:15.975696 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:15.975632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:15.975892 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:15.975851 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:15.975954 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:15.975916 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs podName:e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d nodeName:}" failed. No retries permitted until 2026-07-06 22:20:19.975898218 +0000 UTC m=+10.077625874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs") pod "network-metrics-daemon-xl9br" (UID: "e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:16.076696 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:16.076657 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:16.076868 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:16.076842 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:20:16.076868 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:16.076865 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:20:16.076994 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:16.076878 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zk4lv for pod openshift-network-diagnostics/network-check-target-8hnkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:16.076994 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:16.076939 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv podName:edfcc999-de23-48fe-8d7d-72fb9836aa2b nodeName:}" failed. No retries permitted until 2026-07-06 22:20:20.07691953 +0000 UTC m=+10.178647175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk4lv" (UniqueName: "kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv") pod "network-check-target-8hnkk" (UID: "edfcc999-de23-48fe-8d7d-72fb9836aa2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:16.374068 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:16.373582 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:16.374068 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:16.373720 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:16.374313 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:16.374125 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:16.374313 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:16.374219 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:16.374313 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:16.374239 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:16.374482 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:16.374328 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:18.373083 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:18.373000 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:18.373553 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:18.373148 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:18.373553 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:18.373006 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:18.373553 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:18.373458 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:18.373717 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:18.373549 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:18.373717 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:18.373636 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:19.910142 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:19.910081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:19.910658 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:19.910282 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:19.910658 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:19.910348 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret podName:d1befd3e-57bc-4d94-9563-ef9479a3d10c nodeName:}" failed. No retries permitted until 2026-07-06 22:20:27.910328879 +0000 UTC m=+18.012056534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret") pod "global-pull-secret-syncer-wmpgs" (UID: "d1befd3e-57bc-4d94-9563-ef9479a3d10c") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:20.011570 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:20.011530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:20.011758 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:20.011684 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:20.011758 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:20.011756 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs podName:e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d nodeName:}" failed. No retries permitted until 2026-07-06 22:20:28.011740267 +0000 UTC m=+18.113467910 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs") pod "network-metrics-daemon-xl9br" (UID: "e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:20.112438 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:20.112251 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:20.112438 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:20.112435 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:20:20.112658 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:20.112454 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:20:20.112658 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:20.112466 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zk4lv for pod openshift-network-diagnostics/network-check-target-8hnkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:20.112658 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:20.112525 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv podName:edfcc999-de23-48fe-8d7d-72fb9836aa2b nodeName:}" failed. No retries permitted until 2026-07-06 22:20:28.112509236 +0000 UTC m=+18.214236889 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk4lv" (UniqueName: "kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv") pod "network-check-target-8hnkk" (UID: "edfcc999-de23-48fe-8d7d-72fb9836aa2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:20.374588 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:20.374520 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:20.374785 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:20.374652 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:20.374849 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:20.374823 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:20.374938 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:20.374920 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:20.375018 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:20.374974 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:20.375065 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:20.375049 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:22.373263 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:22.372776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:22.373263 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:22.372824 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:22.373263 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:22.372776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:22.373263 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:22.372932 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:22.373263 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:22.373011 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:22.373263 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:22.373087 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:24.373221 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:24.373182 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:24.373656 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:24.373182 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:24.373656 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:24.373325 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:24.373656 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:24.373182 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:24.373656 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:24.373440 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:24.373656 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:24.373541 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:26.373237 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:26.373202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:26.373692 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:26.373202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:26.373692 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:26.373316 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:26.373692 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:26.373400 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:26.373692 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:26.373202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:26.373692 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:26.373538 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:27.973241 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:27.973200 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:27.973741 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:27.973409 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:27.973741 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:27.973494 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret podName:d1befd3e-57bc-4d94-9563-ef9479a3d10c nodeName:}" failed. No retries permitted until 2026-07-06 22:20:43.973472562 +0000 UTC m=+34.075200220 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret") pod "global-pull-secret-syncer-wmpgs" (UID: "d1befd3e-57bc-4d94-9563-ef9479a3d10c") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:28.073985 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:28.073948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:28.074179 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:28.074098 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:28.074238 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:28.074191 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs podName:e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d nodeName:}" failed. No retries permitted until 2026-07-06 22:20:44.074171193 +0000 UTC m=+34.175898837 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs") pod "network-metrics-daemon-xl9br" (UID: "e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:28.174831 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:28.174796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:28.174974 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:28.174934 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:20:28.174974 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:28.174954 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:20:28.174974 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:28.174964 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zk4lv for pod openshift-network-diagnostics/network-check-target-8hnkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:28.175124 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:28.175012 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv podName:edfcc999-de23-48fe-8d7d-72fb9836aa2b nodeName:}" failed. No retries permitted until 2026-07-06 22:20:44.174997393 +0000 UTC m=+34.276725033 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk4lv" (UniqueName: "kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv") pod "network-check-target-8hnkk" (UID: "edfcc999-de23-48fe-8d7d-72fb9836aa2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:28.373210 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:28.373177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:28.373380 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:28.373177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:28.373380 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:28.373324 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:28.373514 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:28.373417 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:28.373514 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:28.373179 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:28.373618 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:28.373517 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:30.373853 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:30.373817 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:30.374420 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:30.373919 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:30.374420 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:30.373949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:30.374420 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:30.373973 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:30.374420 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:30.374055 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:30.374420 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:30.374089 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:32.373280 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.372949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:32.373954 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.372949 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:32.373954 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:32.373362 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:32.373954 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.372950 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:32.373954 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:32.373444 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:32.373954 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:32.373552 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:32.475768 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.475735 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6n28g" event={"ID":"7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0","Type":"ContainerStarted","Data":"742318cca10ad85dfdc35412dd277b5da15843511017443cfc3b1b16009ccac4"} Jul 06 22:20:32.477258 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.477231 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" event={"ID":"13f5b744-1211-4dab-bf92-1b298db35e5a","Type":"ContainerStarted","Data":"33f6c30afa8b27f8767d8edcad8c25401bf917ee66b1de8b588d8479fd86bdf4"} Jul 06 22:20:32.481630 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.481601 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w9g5x" event={"ID":"5517402c-0226-4bbb-8d38-0200eb62df31","Type":"ContainerStarted","Data":"99cfd57978e677c07b6b8a1ced3d9008900d99cfa6d720f5c25095b6070490cb"} Jul 06 22:20:32.482994 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.482965 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k7phn" event={"ID":"281a0cf8-525d-4171-b0f3-8020a15bfd48","Type":"ContainerStarted","Data":"dcde6d2e509ac88d7816cda66d3b5c088fc9eee1b0de0ee3bcb1cd684c3f8b5e"} Jul 06 22:20:32.484299 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.484275 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" event={"ID":"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa","Type":"ContainerStarted","Data":"2a0c4033f51879446ba67e75095d8a946e3d606f506606f65627751a83f4f807"} Jul 06 22:20:32.485991 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.485972 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" event={"ID":"de18a133-ab7e-4483-a93a-bbdee1fbc449","Type":"ContainerStarted","Data":"6f87dacaa2ec697773a3f99de58e2787b61055f606aacbf64cc7020882bf810e"} Jul 06 22:20:32.486063 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.486029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" event={"ID":"de18a133-ab7e-4483-a93a-bbdee1fbc449","Type":"ContainerStarted","Data":"fdb0f83d7d8ef367435f823d779ae3818be6c3e8cbd9e36e5c2c9544af30e1a3"} Jul 06 22:20:32.486063 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.486059 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" event={"ID":"de18a133-ab7e-4483-a93a-bbdee1fbc449","Type":"ContainerStarted","Data":"ad7061ef0b136f6458ac311a7f8e312a7f0a1c8478e30cd77c4ddc3119e060ff"} Jul 06 22:20:32.487334 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.487313 2578 generic.go:356] "Generic (PLEG): container finished" podID="96ce3e1e-19db-4ebc-8d09-8c6a962cefd0" containerID="d5a83c4b7c6a04fdc71015966a3fb4cc27109f2a6f1c5129e6c0059a439d6844" exitCode=0 Jul 06 22:20:32.487415 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.487389 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z29kj" event={"ID":"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0","Type":"ContainerDied","Data":"d5a83c4b7c6a04fdc71015966a3fb4cc27109f2a6f1c5129e6c0059a439d6844"} Jul 06 22:20:32.488658 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.488637 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gzgkl" event={"ID":"690ba9a7-5ee6-4aeb-93ea-5e087ae83a55","Type":"ContainerStarted","Data":"90e13f1c4cf3cad17580e0c369175be10e7026c9bba9cd02d243fab854629f69"} Jul 06 22:20:32.494848 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.494765 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6n28g" podStartSLOduration=8.437077945 podStartE2EDuration="22.494753889s" totalImagesPullingTime="14.057675944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:32.49445128 +0000 UTC m=+22.596178942" watchObservedRunningTime="2026-07-06 22:20:32.494753889 +0000 UTC m=+22.596481550" Jul 06 22:20:32.525837 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.525767 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k7phn" podStartSLOduration=3.927998171 podStartE2EDuration="22.52575059s" totalImagesPullingTime="18.597752419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:32.510636669 +0000 UTC m=+22.612364331" watchObservedRunningTime="2026-07-06 22:20:32.52575059 +0000 UTC m=+22.627478246" Jul 06 22:20:32.526246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.526222 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w9g5x" podStartSLOduration=3.920592481 podStartE2EDuration="22.526212316s" totalImagesPullingTime="18.605619835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:32.526048213 +0000 UTC m=+22.627775876" watchObservedRunningTime="2026-07-06 22:20:32.526212316 +0000 UTC m=+22.627939979" Jul 06 22:20:32.569460 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.569413 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gzgkl" podStartSLOduration=3.900518214 podStartE2EDuration="22.569392835s" totalImagesPullingTime="18.668874621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:32.569083862 +0000 UTC m=+22.670811523" watchObservedRunningTime="2026-07-06 22:20:32.569392835 +0000 UTC m=+22.671120501" Jul 06 22:20:32.785022 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.784984 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:32.785827 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.785712 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:32.807935 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:32.807888 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rbbsg" podStartSLOduration=4.162851908 podStartE2EDuration="22.80787394s" totalImagesPullingTime="18.645022032s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:32.587224067 +0000 UTC m=+22.688951740" watchObservedRunningTime="2026-07-06 22:20:32.80787394 +0000 UTC m=+22.909601603" Jul 06 22:20:33.494694 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:33.494658 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" event={"ID":"de18a133-ab7e-4483-a93a-bbdee1fbc449","Type":"ContainerStarted","Data":"57836a0000a8f06ad046a055f1f49e605d4f80dda0bf0673572bfac76465e646"} Jul 06 22:20:33.495249 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:33.494748 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" event={"ID":"de18a133-ab7e-4483-a93a-bbdee1fbc449","Type":"ContainerStarted","Data":"e95eff121815d4bc3ef6ad248bc7d70591484237bc996865238cf91e9df5ef2a"} Jul 06 22:20:33.495249 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:33.494789 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" event={"ID":"de18a133-ab7e-4483-a93a-bbdee1fbc449","Type":"ContainerStarted","Data":"c2c4978914d01bbce18fd10607ef6d983c9bf398bdfac2412c8391076efc7845"} Jul 06 22:20:33.496455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:33.496413 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q77mn" event={"ID":"15174860-7d96-48c1-8616-5ddabcff08e8","Type":"ContainerStarted","Data":"4c6ef17f143e65eea9ddc0759bd1c53285b81431522cdf0ff63f3b2b9bc7d259"} Jul 06 22:20:33.496878 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:33.496825 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:33.498399 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:33.498373 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6n28g" Jul 06 22:20:33.517607 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:33.517556 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q77mn" podStartSLOduration=9.462665852 podStartE2EDuration="23.517535413s" totalImagesPullingTime="14.054869561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:33.516918973 +0000 UTC m=+23.618646633" watchObservedRunningTime="2026-07-06 22:20:33.517535413 +0000 UTC m=+23.619263079" Jul 06 22:20:33.818150 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:33.818104 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 22:20:34.293632 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:34.293537 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T22:20:33.818142109Z","UUID":"96758fff-58cd-48d1-83be-3c147e8453d5","Handler":null,"Name":"","Endpoint":""} Jul 06 22:20:34.294997 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:34.294979 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 22:20:34.295129 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:34.295016 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 22:20:34.373682 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:34.373592 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:34.373682 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:34.373630 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:34.373990 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:34.373731 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:34.373990 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:34.373731 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:34.373990 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:34.373879 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:34.373990 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:34.373947 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:34.500915 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:34.500835 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" event={"ID":"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa","Type":"ContainerStarted","Data":"a3217fb6b8a36f3d0e2eb1219fe7ed4c10cb30d7d0167cb30d2323dd4a1f9c5e"} Jul 06 22:20:35.505503 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:35.505423 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" event={"ID":"dd5fd205-1e42-4ce6-8f6e-ca04f80384aa","Type":"ContainerStarted","Data":"c723b2d5c9241b7b5e2a7271d673bc2382e24e1219a56b36c46e417d8d5ee7e2"} Jul 06 22:20:35.508557 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:35.508535 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" event={"ID":"de18a133-ab7e-4483-a93a-bbdee1fbc449","Type":"ContainerStarted","Data":"c3eaa872d7a18bf071950f559fecba79561f98289c0e87f04d088bf96544481c"} Jul 06 22:20:35.527558 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:35.527513 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xmzgj" podStartSLOduration=4.072731869 podStartE2EDuration="25.527493481s" totalImagesPullingTime="21.454761612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:35.527019568 +0000 UTC m=+25.628747231" watchObservedRunningTime="2026-07-06 22:20:35.527493481 +0000 UTC m=+25.629221144" Jul 06 22:20:36.373008 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:36.372976 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:36.373008 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:36.372997 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:36.373285 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:36.372976 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:36.373285 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:36.373102 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:36.373285 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:36.373215 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:36.373437 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:36.373321 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:38.373181 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.372955 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:38.373746 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.372969 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:38.373746 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:38.373296 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:38.373746 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.373028 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:38.373746 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:38.373354 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:38.373746 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:38.373451 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:38.518539 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.518025 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" event={"ID":"de18a133-ab7e-4483-a93a-bbdee1fbc449","Type":"ContainerStarted","Data":"c7cdd504324ecec0bfecd8877413ab98db6e0ba8cfff99a1854b12e36f9f9f77"} Jul 06 22:20:38.518539 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.518484 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:38.518673 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.518657 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:38.518855 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.518811 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:38.534531 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.534506 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:38.535558 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.535537 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:20:38.553568 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:38.553521 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" podStartSLOduration=9.83664049 podStartE2EDuration="28.553507087s" totalImagesPullingTime="18.716866597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:38.552047118 +0000 UTC m=+28.653774781" watchObservedRunningTime="2026-07-06 22:20:38.553507087 +0000 UTC m=+28.655234749" Jul 06 22:20:39.522499 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:39.522462 2578 generic.go:356] "Generic (PLEG): container finished" podID="96ce3e1e-19db-4ebc-8d09-8c6a962cefd0" containerID="3c24627e8f790076c94707859cb7587bed61522a15791aeb8fe7855c6facd77a" exitCode=0 Jul 06 22:20:39.525056 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:39.525029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z29kj" event={"ID":"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0","Type":"ContainerDied","Data":"3c24627e8f790076c94707859cb7587bed61522a15791aeb8fe7855c6facd77a"} Jul 06 22:20:40.373593 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:40.373414 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:40.373763 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:40.373488 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:40.373763 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:40.373517 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:40.373763 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:40.373746 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:40.373890 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:40.373650 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:40.373890 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:40.373830 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:40.569391 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:40.569345 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wmpgs"] Jul 06 22:20:40.569774 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:40.569515 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:40.569774 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:40.569626 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:40.570464 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:40.570139 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8hnkk"] Jul 06 22:20:40.570464 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:40.570228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:40.570464 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:40.570312 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:40.570828 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:40.570807 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xl9br"] Jul 06 22:20:40.570918 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:40.570906 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:40.571025 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:40.571003 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:41.530975 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:41.530895 2578 generic.go:356] "Generic (PLEG): container finished" podID="96ce3e1e-19db-4ebc-8d09-8c6a962cefd0" containerID="202eea9210b2a506cdf98773ea4787cd44e8eee6a6318080e2dad6508dc27d2c" exitCode=0 Jul 06 22:20:41.530975 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:41.530939 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z29kj" event={"ID":"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0","Type":"ContainerDied","Data":"202eea9210b2a506cdf98773ea4787cd44e8eee6a6318080e2dad6508dc27d2c"} Jul 06 22:20:42.373023 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:42.372984 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:42.373023 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:42.373015 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:42.373553 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:42.372988 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:42.373553 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:42.373089 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:42.373553 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:42.373165 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:42.373553 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:42.373223 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:42.536247 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:42.536060 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z29kj" event={"ID":"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0","Type":"ContainerStarted","Data":"8c2f9fba2cd542a8c0b4080db48d4bed356ec5a90881d4c3608614c9f8c353ca"} Jul 06 22:20:43.540541 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:43.540507 2578 generic.go:356] "Generic (PLEG): container finished" podID="96ce3e1e-19db-4ebc-8d09-8c6a962cefd0" containerID="8c2f9fba2cd542a8c0b4080db48d4bed356ec5a90881d4c3608614c9f8c353ca" exitCode=0 Jul 06 22:20:43.540958 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:43.540561 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z29kj" event={"ID":"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0","Type":"ContainerDied","Data":"8c2f9fba2cd542a8c0b4080db48d4bed356ec5a90881d4c3608614c9f8c353ca"} Jul 06 22:20:43.983969 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:43.983777 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:43.984148 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:43.983920 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:43.984148 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:43.984057 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret podName:d1befd3e-57bc-4d94-9563-ef9479a3d10c nodeName:}" failed. No retries permitted until 2026-07-06 22:21:15.984042233 +0000 UTC m=+66.085769876 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret") pod "global-pull-secret-syncer-wmpgs" (UID: "d1befd3e-57bc-4d94-9563-ef9479a3d10c") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:20:44.084709 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.084676 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:44.084863 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.084844 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:44.084935 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.084921 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs podName:e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d nodeName:}" failed. No retries permitted until 2026-07-06 22:21:16.084899863 +0000 UTC m=+66.186627525 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs") pod "network-metrics-daemon-xl9br" (UID: "e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:20:44.185335 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.185301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:44.185492 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.185472 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:20:44.185492 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.185489 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:20:44.185582 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.185500 2578 projected.go:196] Error preparing data for projected volume kube-api-access-zk4lv for pod openshift-network-diagnostics/network-check-target-8hnkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:44.185582 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.185554 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv podName:edfcc999-de23-48fe-8d7d-72fb9836aa2b nodeName:}" failed. No retries permitted until 2026-07-06 22:21:16.185537092 +0000 UTC m=+66.287264754 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zk4lv" (UniqueName: "kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv") pod "network-check-target-8hnkk" (UID: "edfcc999-de23-48fe-8d7d-72fb9836aa2b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:20:44.373469 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.373435 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:44.373645 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.373475 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:44.373645 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.373565 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8hnkk" podUID="edfcc999-de23-48fe-8d7d-72fb9836aa2b" Jul 06 22:20:44.373645 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.373634 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:44.373804 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.373756 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xl9br" podUID="e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d" Jul 06 22:20:44.373887 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.373861 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wmpgs" podUID="d1befd3e-57bc-4d94-9563-ef9479a3d10c" Jul 06 22:20:44.600980 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.600955 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-114.ec2.internal" event="NodeReady" Jul 06 22:20:44.601353 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.601074 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 22:20:44.654630 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.654477 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n2j6t"] Jul 06 22:20:44.657258 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.657235 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-285wd"] Jul 06 22:20:44.657431 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.657352 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.659568 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.659547 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-285wd" Jul 06 22:20:44.659746 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.659724 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2q2fk\"" Jul 06 22:20:44.659841 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.659745 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 22:20:44.660030 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.660013 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 22:20:44.661754 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.661738 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gmtbp\"" Jul 06 22:20:44.661885 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.661870 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 22:20:44.661934 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.661883 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 22:20:44.662186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.662170 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 22:20:44.669743 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.669168 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-285wd"] Jul 06 22:20:44.669743 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.669693 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n2j6t"] Jul 06 22:20:44.789315 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.789280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd4v7\" (UniqueName: \"kubernetes.io/projected/e2a31111-3a9d-4f49-87fd-f0f417156ee7-kube-api-access-jd4v7\") pod \"ingress-canary-285wd\" (UID: \"e2a31111-3a9d-4f49-87fd-f0f417156ee7\") " pod="openshift-ingress-canary/ingress-canary-285wd" Jul 06 22:20:44.789315 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.789318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qqbb\" (UniqueName: \"kubernetes.io/projected/224f9e83-3e16-471f-9e66-1ef1b08343c4-kube-api-access-4qqbb\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.789538 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.789341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/224f9e83-3e16-471f-9e66-1ef1b08343c4-tmp-dir\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.789538 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.789368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2a31111-3a9d-4f49-87fd-f0f417156ee7-cert\") pod \"ingress-canary-285wd\" (UID: \"e2a31111-3a9d-4f49-87fd-f0f417156ee7\") " pod="openshift-ingress-canary/ingress-canary-285wd" Jul 06 22:20:44.789538 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.789388 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/224f9e83-3e16-471f-9e66-1ef1b08343c4-config-volume\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.789538 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.789462 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.817793 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.817761 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz"] Jul 06 22:20:44.822305 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.822282 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz" Jul 06 22:20:44.824415 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.824396 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 22:20:44.824542 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.824524 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vfld2\"" Jul 06 22:20:44.824752 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.824738 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 22:20:44.832681 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.832656 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz"] Jul 06 22:20:44.890506 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.890468 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd4v7\" (UniqueName: \"kubernetes.io/projected/e2a31111-3a9d-4f49-87fd-f0f417156ee7-kube-api-access-jd4v7\") pod \"ingress-canary-285wd\" (UID: \"e2a31111-3a9d-4f49-87fd-f0f417156ee7\") " pod="openshift-ingress-canary/ingress-canary-285wd" Jul 06 22:20:44.890506 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.890509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqbb\" (UniqueName: \"kubernetes.io/projected/224f9e83-3e16-471f-9e66-1ef1b08343c4-kube-api-access-4qqbb\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.890722 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.890532 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/224f9e83-3e16-471f-9e66-1ef1b08343c4-tmp-dir\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.890722 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.890563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2a31111-3a9d-4f49-87fd-f0f417156ee7-cert\") pod \"ingress-canary-285wd\" (UID: \"e2a31111-3a9d-4f49-87fd-f0f417156ee7\") " pod="openshift-ingress-canary/ingress-canary-285wd" Jul 06 22:20:44.890821 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.890743 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/224f9e83-3e16-471f-9e66-1ef1b08343c4-config-volume\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.890821 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.890790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.891174 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.890881 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/224f9e83-3e16-471f-9e66-1ef1b08343c4-tmp-dir\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.891174 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.890951 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:20:44.891174 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:44.891016 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls podName:224f9e83-3e16-471f-9e66-1ef1b08343c4 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:45.390998394 +0000 UTC m=+35.492726051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls") pod "dns-default-n2j6t" (UID: "224f9e83-3e16-471f-9e66-1ef1b08343c4") : secret "dns-default-metrics-tls" not found Jul 06 22:20:44.891347 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.891302 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/224f9e83-3e16-471f-9e66-1ef1b08343c4-config-volume\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.895609 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.895581 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2a31111-3a9d-4f49-87fd-f0f417156ee7-cert\") pod \"ingress-canary-285wd\" (UID: \"e2a31111-3a9d-4f49-87fd-f0f417156ee7\") " pod="openshift-ingress-canary/ingress-canary-285wd" Jul 06 22:20:44.902319 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.902269 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qqbb\" (UniqueName: \"kubernetes.io/projected/224f9e83-3e16-471f-9e66-1ef1b08343c4-kube-api-access-4qqbb\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:44.902421 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.902333 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd4v7\" (UniqueName: \"kubernetes.io/projected/e2a31111-3a9d-4f49-87fd-f0f417156ee7-kube-api-access-jd4v7\") pod \"ingress-canary-285wd\" (UID: \"e2a31111-3a9d-4f49-87fd-f0f417156ee7\") " pod="openshift-ingress-canary/ingress-canary-285wd" Jul 06 22:20:44.977027 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.976953 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-285wd" Jul 06 22:20:44.992348 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:44.992308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv7qn\" (UniqueName: \"kubernetes.io/projected/109d75df-07a3-4c87-908b-807ac08b1e1b-kube-api-access-rv7qn\") pod \"migrator-d57d58cc9-l2wzz\" (UID: \"109d75df-07a3-4c87-908b-807ac08b1e1b\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz" Jul 06 22:20:45.093696 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.093662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv7qn\" (UniqueName: \"kubernetes.io/projected/109d75df-07a3-4c87-908b-807ac08b1e1b-kube-api-access-rv7qn\") pod \"migrator-d57d58cc9-l2wzz\" (UID: \"109d75df-07a3-4c87-908b-807ac08b1e1b\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz" Jul 06 22:20:45.104215 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.104162 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv7qn\" (UniqueName: \"kubernetes.io/projected/109d75df-07a3-4c87-908b-807ac08b1e1b-kube-api-access-rv7qn\") pod \"migrator-d57d58cc9-l2wzz\" (UID: \"109d75df-07a3-4c87-908b-807ac08b1e1b\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz" Jul 06 22:20:45.127929 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.127898 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-285wd"] Jul 06 22:20:45.131503 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.131480 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz" Jul 06 22:20:45.132218 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:45.132196 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2a31111_3a9d_4f49_87fd_f0f417156ee7.slice/crio-8deab00031b5e627a2101c5b907533d677ad19b27172982a4998a10d7447091e WatchSource:0}: Error finding container 8deab00031b5e627a2101c5b907533d677ad19b27172982a4998a10d7447091e: Status 404 returned error can't find the container with id 8deab00031b5e627a2101c5b907533d677ad19b27172982a4998a10d7447091e Jul 06 22:20:45.176493 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.176462 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9vz26"] Jul 06 22:20:45.182381 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.182359 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.186095 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.186033 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 22:20:45.186095 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.186062 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 22:20:45.188213 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.187367 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 22:20:45.190146 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.188360 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 22:20:45.192648 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.190774 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dhrwd\"" Jul 06 22:20:45.195636 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.195300 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9vz26"] Jul 06 22:20:45.267907 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.267827 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz"] Jul 06 22:20:45.271095 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:45.271063 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109d75df_07a3_4c87_908b_807ac08b1e1b.slice/crio-ce6d1e0248cbdbd0b2bdc2b74155045518382a8863f0d5752802e2eccb3121b5 WatchSource:0}: Error finding container ce6d1e0248cbdbd0b2bdc2b74155045518382a8863f0d5752802e2eccb3121b5: Status 404 returned error can't find the container with id ce6d1e0248cbdbd0b2bdc2b74155045518382a8863f0d5752802e2eccb3121b5 Jul 06 22:20:45.295443 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.295412 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.295587 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.295449 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/02417ea4-0e61-4d6e-9423-487aa1f2a053-crio-socket\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.295587 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.295491 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/02417ea4-0e61-4d6e-9423-487aa1f2a053-data-volume\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.295587 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.295538 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.295587 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.295572 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjp9l\" (UniqueName: \"kubernetes.io/projected/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-api-access-cjp9l\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.396092 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.396051 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:45.396293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.396105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/02417ea4-0e61-4d6e-9423-487aa1f2a053-data-volume\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.396293 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.396201 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:20:45.396293 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.396258 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:20:45.396293 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.396207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.396293 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.396276 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls podName:224f9e83-3e16-471f-9e66-1ef1b08343c4 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:46.396253577 +0000 UTC m=+36.497981220 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls") pod "dns-default-n2j6t" (UID: "224f9e83-3e16-471f-9e66-1ef1b08343c4") : secret "dns-default-metrics-tls" not found Jul 06 22:20:45.396545 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.396319 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:45.896302913 +0000 UTC m=+35.998030573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : configmap "kube-rbac-proxy" not found Jul 06 22:20:45.396545 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.396313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjp9l\" (UniqueName: \"kubernetes.io/projected/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-api-access-cjp9l\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.396545 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.396381 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.396545 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.396418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/02417ea4-0e61-4d6e-9423-487aa1f2a053-crio-socket\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.396545 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.396493 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/02417ea4-0e61-4d6e-9423-487aa1f2a053-data-volume\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.396545 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.396535 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:20:45.396802 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.396580 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:45.896567098 +0000 UTC m=+35.998294738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : secret "insights-runtime-extractor-tls" not found Jul 06 22:20:45.396802 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.396649 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/02417ea4-0e61-4d6e-9423-487aa1f2a053-crio-socket\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.405713 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.405682 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjp9l\" (UniqueName: \"kubernetes.io/projected/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-api-access-cjp9l\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.545240 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.545141 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz" event={"ID":"109d75df-07a3-4c87-908b-807ac08b1e1b","Type":"ContainerStarted","Data":"ce6d1e0248cbdbd0b2bdc2b74155045518382a8863f0d5752802e2eccb3121b5"} Jul 06 22:20:45.546204 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.546181 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-285wd" event={"ID":"e2a31111-3a9d-4f49-87fd-f0f417156ee7","Type":"ContainerStarted","Data":"8deab00031b5e627a2101c5b907533d677ad19b27172982a4998a10d7447091e"} Jul 06 22:20:45.901599 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.901324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.901599 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:45.901403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:45.901599 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.901419 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:20:45.901599 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.901481 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:46.901463969 +0000 UTC m=+37.003191626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : configmap "kube-rbac-proxy" not found Jul 06 22:20:45.901599 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.901538 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:20:45.901599 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:45.901574 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:46.901563415 +0000 UTC m=+37.003291059 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : secret "insights-runtime-extractor-tls" not found Jul 06 22:20:46.373584 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.372744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:20:46.373584 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.372808 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:20:46.373584 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.372816 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:20:46.377968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.377744 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 22:20:46.377968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.377783 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t8ckr\"" Jul 06 22:20:46.378209 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.378140 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 22:20:46.378684 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.378501 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 22:20:46.378684 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.378502 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 22:20:46.380639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.380617 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zplcn\"" Jul 06 22:20:46.406458 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.406431 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:46.406630 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:46.406613 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:20:46.406697 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:46.406678 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls podName:224f9e83-3e16-471f-9e66-1ef1b08343c4 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:48.40666151 +0000 UTC m=+38.508389153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls") pod "dns-default-n2j6t" (UID: "224f9e83-3e16-471f-9e66-1ef1b08343c4") : secret "dns-default-metrics-tls" not found Jul 06 22:20:46.911441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.911336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:46.911918 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:46.911490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:46.911918 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:46.911529 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:20:46.911918 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:46.911590 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:20:46.911918 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:46.911612 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:48.911587722 +0000 UTC m=+39.013315364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : secret "insights-runtime-extractor-tls" not found Jul 06 22:20:46.911918 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:46.911640 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:48.91162643 +0000 UTC m=+39.013354083 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : configmap "kube-rbac-proxy" not found Jul 06 22:20:47.553908 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:47.553871 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz" event={"ID":"109d75df-07a3-4c87-908b-807ac08b1e1b","Type":"ContainerStarted","Data":"58b789f9262c43560a49ca396933964f076120aa279d9da0d938e29aa6c7d742"} Jul 06 22:20:47.554093 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:47.553935 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz" event={"ID":"109d75df-07a3-4c87-908b-807ac08b1e1b","Type":"ContainerStarted","Data":"9efc8bdefe193999fed5ead5a909d228656c9d90cf496d0428b2561ef0a4d74b"} Jul 06 22:20:47.572596 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:47.572522 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-l2wzz" podStartSLOduration=2.313401337 podStartE2EDuration="3.572503987s" totalImagesPullingTime="1.25910265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:47.571623549 +0000 UTC m=+37.673351212" watchObservedRunningTime="2026-07-06 22:20:47.572503987 +0000 UTC m=+37.674231629" Jul 06 22:20:47.982731 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:47.982702 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k7phn_281a0cf8-525d-4171-b0f3-8020a15bfd48/dns-node-resolver/0.log" Jul 06 22:20:48.423517 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:48.423481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:48.423734 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:48.423666 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:20:48.423797 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:48.423762 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls podName:224f9e83-3e16-471f-9e66-1ef1b08343c4 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:52.423740056 +0000 UTC m=+42.525467716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls") pod "dns-default-n2j6t" (UID: "224f9e83-3e16-471f-9e66-1ef1b08343c4") : secret "dns-default-metrics-tls" not found Jul 06 22:20:48.927522 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:48.927488 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:48.927739 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:48.927607 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:48.927739 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:48.927681 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:20:48.927739 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:48.927714 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:20:48.927873 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:48.927769 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:52.927748256 +0000 UTC m=+43.029475900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : secret "insights-runtime-extractor-tls" not found Jul 06 22:20:48.927873 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:48.927789 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:20:52.92777945 +0000 UTC m=+43.029507090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : configmap "kube-rbac-proxy" not found Jul 06 22:20:49.190677 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:49.190590 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w9g5x_5517402c-0226-4bbb-8d38-0200eb62df31/node-ca/0.log" Jul 06 22:20:50.182993 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.182916 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-l2wzz_109d75df-07a3-4c87-908b-807ac08b1e1b/migrator/0.log" Jul 06 22:20:50.367299 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.367267 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-kgj49"] Jul 06 22:20:50.384060 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.384035 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-l2wzz_109d75df-07a3-4c87-908b-807ac08b1e1b/graceful-termination/0.log" Jul 06 22:20:50.398525 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.398489 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-kgj49"] Jul 06 22:20:50.398672 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.398623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.401128 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.401092 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4hhzd\"" Jul 06 22:20:50.401247 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.401154 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 22:20:50.401247 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.401096 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 22:20:50.401810 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.401797 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 22:20:50.401918 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.401898 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 22:20:50.539273 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.539176 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da4a1303-958b-41d6-815f-81f4ddaf7d14-signing-key\") pod \"service-ca-758f599c5-kgj49\" (UID: \"da4a1303-958b-41d6-815f-81f4ddaf7d14\") " pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.539273 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.539236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74gtv\" (UniqueName: \"kubernetes.io/projected/da4a1303-958b-41d6-815f-81f4ddaf7d14-kube-api-access-74gtv\") pod \"service-ca-758f599c5-kgj49\" (UID: \"da4a1303-958b-41d6-815f-81f4ddaf7d14\") " pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.539460 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.539307 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da4a1303-958b-41d6-815f-81f4ddaf7d14-signing-cabundle\") pod \"service-ca-758f599c5-kgj49\" (UID: \"da4a1303-958b-41d6-815f-81f4ddaf7d14\") " pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.564284 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.564242 2578 generic.go:356] "Generic (PLEG): container finished" podID="96ce3e1e-19db-4ebc-8d09-8c6a962cefd0" containerID="4c9096d902f22bcf5c6d31368524b2be86306f414eaf29457540d34fe0baa044" exitCode=0 Jul 06 22:20:50.564471 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.564288 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z29kj" event={"ID":"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0","Type":"ContainerDied","Data":"4c9096d902f22bcf5c6d31368524b2be86306f414eaf29457540d34fe0baa044"} Jul 06 22:20:50.565912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.565882 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-285wd" event={"ID":"e2a31111-3a9d-4f49-87fd-f0f417156ee7","Type":"ContainerStarted","Data":"2379c4b250e0db4d9c23b1577427fcffe7fdc6c4451d769606dacfe36a9f2aa8"} Jul 06 22:20:50.603933 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.603871 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-285wd" podStartSLOduration=2.068690067 podStartE2EDuration="6.60384621s" totalImagesPullingTime="4.535156143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:50.603015787 +0000 UTC m=+40.704743463" watchObservedRunningTime="2026-07-06 22:20:50.60384621 +0000 UTC m=+40.705573873" Jul 06 22:20:50.639951 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.639919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da4a1303-958b-41d6-815f-81f4ddaf7d14-signing-key\") pod \"service-ca-758f599c5-kgj49\" (UID: \"da4a1303-958b-41d6-815f-81f4ddaf7d14\") " pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.640138 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.639986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74gtv\" (UniqueName: \"kubernetes.io/projected/da4a1303-958b-41d6-815f-81f4ddaf7d14-kube-api-access-74gtv\") pod \"service-ca-758f599c5-kgj49\" (UID: \"da4a1303-958b-41d6-815f-81f4ddaf7d14\") " pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.640138 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.640017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da4a1303-958b-41d6-815f-81f4ddaf7d14-signing-cabundle\") pod \"service-ca-758f599c5-kgj49\" (UID: \"da4a1303-958b-41d6-815f-81f4ddaf7d14\") " pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.640759 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.640730 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da4a1303-958b-41d6-815f-81f4ddaf7d14-signing-cabundle\") pod \"service-ca-758f599c5-kgj49\" (UID: \"da4a1303-958b-41d6-815f-81f4ddaf7d14\") " pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.643853 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.643829 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da4a1303-958b-41d6-815f-81f4ddaf7d14-signing-key\") pod \"service-ca-758f599c5-kgj49\" (UID: \"da4a1303-958b-41d6-815f-81f4ddaf7d14\") " pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.649985 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.649958 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74gtv\" (UniqueName: \"kubernetes.io/projected/da4a1303-958b-41d6-815f-81f4ddaf7d14-kube-api-access-74gtv\") pod \"service-ca-758f599c5-kgj49\" (UID: \"da4a1303-958b-41d6-815f-81f4ddaf7d14\") " pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.708040 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.708012 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-kgj49" Jul 06 22:20:50.876004 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:50.875964 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-kgj49"] Jul 06 22:20:50.889909 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:20:50.889880 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda4a1303_958b_41d6_815f_81f4ddaf7d14.slice/crio-a7df7de9d63a227088e19e62ffdd66a15bb617d59a29493d1ee14676ace59aba WatchSource:0}: Error finding container a7df7de9d63a227088e19e62ffdd66a15bb617d59a29493d1ee14676ace59aba: Status 404 returned error can't find the container with id a7df7de9d63a227088e19e62ffdd66a15bb617d59a29493d1ee14676ace59aba Jul 06 22:20:51.572213 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:51.572174 2578 generic.go:356] "Generic (PLEG): container finished" podID="96ce3e1e-19db-4ebc-8d09-8c6a962cefd0" containerID="b192da82dc1405a2ae81eaeb9b321d3d19d47e2cf254c74495a1fdeeab910a95" exitCode=0 Jul 06 22:20:51.572697 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:51.572265 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z29kj" event={"ID":"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0","Type":"ContainerDied","Data":"b192da82dc1405a2ae81eaeb9b321d3d19d47e2cf254c74495a1fdeeab910a95"} Jul 06 22:20:51.573666 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:51.573640 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-kgj49" event={"ID":"da4a1303-958b-41d6-815f-81f4ddaf7d14","Type":"ContainerStarted","Data":"a7df7de9d63a227088e19e62ffdd66a15bb617d59a29493d1ee14676ace59aba"} Jul 06 22:20:52.457727 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:52.457687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:20:52.457880 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:52.457857 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:20:52.457935 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:52.457922 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls podName:224f9e83-3e16-471f-9e66-1ef1b08343c4 nodeName:}" failed. No retries permitted until 2026-07-06 22:21:00.45790725 +0000 UTC m=+50.559634895 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls") pod "dns-default-n2j6t" (UID: "224f9e83-3e16-471f-9e66-1ef1b08343c4") : secret "dns-default-metrics-tls" not found Jul 06 22:20:52.579532 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:52.579491 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z29kj" event={"ID":"96ce3e1e-19db-4ebc-8d09-8c6a962cefd0","Type":"ContainerStarted","Data":"a7fcecf9b98665da8f933b844bd58f6f58fa5ed699147c52b5baaf50d25dca33"} Jul 06 22:20:52.608127 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:52.608057 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z29kj" podStartSLOduration=10.103623454 podStartE2EDuration="42.60803916s" totalImagesPullingTime="32.504415706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:52.606565807 +0000 UTC m=+42.708293471" watchObservedRunningTime="2026-07-06 22:20:52.60803916 +0000 UTC m=+42.709766823" Jul 06 22:20:52.962487 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:52.962439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:52.962701 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:52.962527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:20:52.962701 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:52.962552 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:20:52.962701 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:52.962642 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:21:00.962618383 +0000 UTC m=+51.064346028 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : configmap "kube-rbac-proxy" not found Jul 06 22:20:52.962701 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:52.962679 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:20:52.962922 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:20:52.962734 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:21:00.962719466 +0000 UTC m=+51.064447122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : secret "insights-runtime-extractor-tls" not found Jul 06 22:20:54.586729 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:54.586691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-kgj49" event={"ID":"da4a1303-958b-41d6-815f-81f4ddaf7d14","Type":"ContainerStarted","Data":"c8c43c0b56a339badb3e4c4453a8e15100667dff2a205b291c22ea98087a3f1e"} Jul 06 22:20:54.605479 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:20:54.605418 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-kgj49" podStartSLOduration=1.897731221 podStartE2EDuration="4.605394394s" totalImagesPullingTime="2.707663173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:20:54.604804743 +0000 UTC m=+44.706532407" watchObservedRunningTime="2026-07-06 22:20:54.605394394 +0000 UTC m=+44.707122041" Jul 06 22:21:00.525029 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:00.524981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:21:00.527348 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:00.527322 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/224f9e83-3e16-471f-9e66-1ef1b08343c4-metrics-tls\") pod \"dns-default-n2j6t\" (UID: \"224f9e83-3e16-471f-9e66-1ef1b08343c4\") " pod="openshift-dns/dns-default-n2j6t" Jul 06 22:21:00.570274 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:00.570233 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n2j6t" Jul 06 22:21:00.724932 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:00.724899 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n2j6t"] Jul 06 22:21:00.728209 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:21:00.728178 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod224f9e83_3e16_471f_9e66_1ef1b08343c4.slice/crio-d317756682466c18f3ff9b6dbe7e33e964f6c3158a4e89aa99e1d05d5e314c6b WatchSource:0}: Error finding container d317756682466c18f3ff9b6dbe7e33e964f6c3158a4e89aa99e1d05d5e314c6b: Status 404 returned error can't find the container with id d317756682466c18f3ff9b6dbe7e33e964f6c3158a4e89aa99e1d05d5e314c6b Jul 06 22:21:01.029439 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:01.029402 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:21:01.029614 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:01.029462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:21:01.029614 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:21:01.029564 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:21:01.029690 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:21:01.029640 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls podName:02417ea4-0e61-4d6e-9423-487aa1f2a053 nodeName:}" failed. No retries permitted until 2026-07-06 22:21:17.029617263 +0000 UTC m=+67.131344902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9vz26" (UID: "02417ea4-0e61-4d6e-9423-487aa1f2a053") : secret "insights-runtime-extractor-tls" not found Jul 06 22:21:01.029946 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:01.029926 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/02417ea4-0e61-4d6e-9423-487aa1f2a053-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:21:01.605706 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:01.605661 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2j6t" event={"ID":"224f9e83-3e16-471f-9e66-1ef1b08343c4","Type":"ContainerStarted","Data":"d317756682466c18f3ff9b6dbe7e33e964f6c3158a4e89aa99e1d05d5e314c6b"} Jul 06 22:21:03.614647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:03.614612 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2j6t" event={"ID":"224f9e83-3e16-471f-9e66-1ef1b08343c4","Type":"ContainerStarted","Data":"df71214830cea18cd0101ebcd9b1c5074ff3b7abcffaed9bf8ab4d6814847d7e"} Jul 06 22:21:03.615052 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:03.614665 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2j6t" event={"ID":"224f9e83-3e16-471f-9e66-1ef1b08343c4","Type":"ContainerStarted","Data":"d52b1f255a507328c46d7d708e8ac9df30fa70624ecb9d54c83b5f2418b0961f"} Jul 06 22:21:03.615052 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:03.614714 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n2j6t" Jul 06 22:21:03.634051 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:03.634000 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n2j6t" podStartSLOduration=17.146201328 podStartE2EDuration="19.633983073s" totalImagesPullingTime="2.487781745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:21:03.632796622 +0000 UTC m=+53.734524283" watchObservedRunningTime="2026-07-06 22:21:03.633983073 +0000 UTC m=+53.735710726" Jul 06 22:21:10.537381 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:10.537350 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ptrf4" Jul 06 22:21:11.695276 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.695245 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd"] Jul 06 22:21:11.731991 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.731958 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5"] Jul 06 22:21:11.732159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.732081 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" Jul 06 22:21:11.735753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.735731 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 22:21:11.736383 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.736362 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 22:21:11.737045 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.737026 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 22:21:11.737799 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.737779 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7wcph\"" Jul 06 22:21:11.739298 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.738849 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 22:21:11.745870 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.745850 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd"] Jul 06 22:21:11.745954 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.745879 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5"] Jul 06 22:21:11.745996 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.745962 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.751154 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.751106 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 22:21:11.751154 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.751142 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 22:21:11.751290 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.751131 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 22:21:11.752159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.752141 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 22:21:11.809517 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.809482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.809710 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.809531 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-hub\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.809710 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.809547 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n944n\" (UniqueName: \"kubernetes.io/projected/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-kube-api-access-n944n\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.809710 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.809620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqdxv\" (UniqueName: \"kubernetes.io/projected/cd134615-79ff-43be-bd76-58972bf51276-kube-api-access-tqdxv\") pod \"managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd\" (UID: \"cd134615-79ff-43be-bd76-58972bf51276\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" Jul 06 22:21:11.809710 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.809668 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cd134615-79ff-43be-bd76-58972bf51276-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd\" (UID: \"cd134615-79ff-43be-bd76-58972bf51276\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" Jul 06 22:21:11.809710 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.809685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.809710 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.809705 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.809908 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.809755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-ca\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.911086 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.911053 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.911291 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.911103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-hub\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.911291 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.911148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n944n\" (UniqueName: \"kubernetes.io/projected/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-kube-api-access-n944n\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.911406 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.911281 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqdxv\" (UniqueName: \"kubernetes.io/projected/cd134615-79ff-43be-bd76-58972bf51276-kube-api-access-tqdxv\") pod \"managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd\" (UID: \"cd134615-79ff-43be-bd76-58972bf51276\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" Jul 06 22:21:11.911406 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.911329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cd134615-79ff-43be-bd76-58972bf51276-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd\" (UID: \"cd134615-79ff-43be-bd76-58972bf51276\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" Jul 06 22:21:11.911406 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.911356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.911406 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.911391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.911601 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.911475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-ca\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.912006 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.911980 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.913771 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.913750 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.913873 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.913773 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-ca\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.913916 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.913892 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-hub\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.914225 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.914209 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cd134615-79ff-43be-bd76-58972bf51276-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd\" (UID: \"cd134615-79ff-43be-bd76-58972bf51276\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" Jul 06 22:21:11.914265 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.914207 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.923100 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.923070 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n944n\" (UniqueName: \"kubernetes.io/projected/5e1f4546-c7c0-4ad9-9cb1-b094b22e4683-kube-api-access-n944n\") pod \"cluster-proxy-proxy-agent-f8b554777-vzlm5\" (UID: \"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:11.923640 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:11.923618 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqdxv\" (UniqueName: \"kubernetes.io/projected/cd134615-79ff-43be-bd76-58972bf51276-kube-api-access-tqdxv\") pod \"managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd\" (UID: \"cd134615-79ff-43be-bd76-58972bf51276\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" Jul 06 22:21:12.058516 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:12.058424 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" Jul 06 22:21:12.065387 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:12.065363 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" Jul 06 22:21:12.192008 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:12.191948 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd"] Jul 06 22:21:12.194503 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:21:12.194474 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd134615_79ff_43be_bd76_58972bf51276.slice/crio-aa955ad1ab57385ec0c78e22bb558646a6838c265ba1dca1fa24d0e754f62677 WatchSource:0}: Error finding container aa955ad1ab57385ec0c78e22bb558646a6838c265ba1dca1fa24d0e754f62677: Status 404 returned error can't find the container with id aa955ad1ab57385ec0c78e22bb558646a6838c265ba1dca1fa24d0e754f62677 Jul 06 22:21:12.211270 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:12.208194 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5"] Jul 06 22:21:12.212412 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:21:12.212382 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e1f4546_c7c0_4ad9_9cb1_b094b22e4683.slice/crio-7d2eacb68c2b5138833ad5fe0a16292b9de5d1d4e8b8d9f73234cb849119101a WatchSource:0}: Error finding container 7d2eacb68c2b5138833ad5fe0a16292b9de5d1d4e8b8d9f73234cb849119101a: Status 404 returned error can't find the container with id 7d2eacb68c2b5138833ad5fe0a16292b9de5d1d4e8b8d9f73234cb849119101a Jul 06 22:21:12.644315 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:12.644269 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" event={"ID":"cd134615-79ff-43be-bd76-58972bf51276","Type":"ContainerStarted","Data":"aa955ad1ab57385ec0c78e22bb558646a6838c265ba1dca1fa24d0e754f62677"} Jul 06 22:21:12.645301 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:12.645279 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" event={"ID":"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683","Type":"ContainerStarted","Data":"7d2eacb68c2b5138833ad5fe0a16292b9de5d1d4e8b8d9f73234cb849119101a"} Jul 06 22:21:13.620879 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:13.620810 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n2j6t" Jul 06 22:21:16.051002 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.050953 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:21:16.053884 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.053863 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 22:21:16.064303 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.064276 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d1befd3e-57bc-4d94-9563-ef9479a3d10c-original-pull-secret\") pod \"global-pull-secret-syncer-wmpgs\" (UID: \"d1befd3e-57bc-4d94-9563-ef9479a3d10c\") " pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:21:16.098313 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.098268 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wmpgs" Jul 06 22:21:16.151764 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.151725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:21:16.154384 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.154357 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 22:21:16.164927 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.164899 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d-metrics-certs\") pod \"network-metrics-daemon-xl9br\" (UID: \"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d\") " pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:21:16.253354 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.253205 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:21:16.255711 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.255685 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 22:21:16.265666 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.265643 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 22:21:16.277158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.277135 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk4lv\" (UniqueName: \"kubernetes.io/projected/edfcc999-de23-48fe-8d7d-72fb9836aa2b-kube-api-access-zk4lv\") pod \"network-check-target-8hnkk\" (UID: \"edfcc999-de23-48fe-8d7d-72fb9836aa2b\") " pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:21:16.382590 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.382566 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wmpgs"] Jul 06 22:21:16.386428 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:21:16.386402 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1befd3e_57bc_4d94_9563_ef9479a3d10c.slice/crio-63e09019ba5ea55b4e7c8addc60e26732dd977d6ec04591999691477292262b0 WatchSource:0}: Error finding container 63e09019ba5ea55b4e7c8addc60e26732dd977d6ec04591999691477292262b0: Status 404 returned error can't find the container with id 63e09019ba5ea55b4e7c8addc60e26732dd977d6ec04591999691477292262b0 Jul 06 22:21:16.391594 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.391402 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t8ckr\"" Jul 06 22:21:16.399220 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.399186 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xl9br" Jul 06 22:21:16.408735 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.408685 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zplcn\"" Jul 06 22:21:16.417372 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.417350 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:21:16.539545 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.539500 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xl9br"] Jul 06 22:21:16.544274 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:21:16.544241 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d8204a_b0f6_4ee1_b0ba_acc52c873c5d.slice/crio-ce69f79188f0a9b62357a84d35b2f1ce38d66cfaf6b7e30d6039da1c2b72e73a WatchSource:0}: Error finding container ce69f79188f0a9b62357a84d35b2f1ce38d66cfaf6b7e30d6039da1c2b72e73a: Status 404 returned error can't find the container with id ce69f79188f0a9b62357a84d35b2f1ce38d66cfaf6b7e30d6039da1c2b72e73a Jul 06 22:21:16.563948 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.563883 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8hnkk"] Jul 06 22:21:16.566822 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:21:16.566798 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedfcc999_de23_48fe_8d7d_72fb9836aa2b.slice/crio-67434afd92482fd5bedceb4b0785d50e1fbf8788d524fcb17660adfd1ba13cb1 WatchSource:0}: Error finding container 67434afd92482fd5bedceb4b0785d50e1fbf8788d524fcb17660adfd1ba13cb1: Status 404 returned error can't find the container with id 67434afd92482fd5bedceb4b0785d50e1fbf8788d524fcb17660adfd1ba13cb1 Jul 06 22:21:16.662372 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.662336 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" event={"ID":"cd134615-79ff-43be-bd76-58972bf51276","Type":"ContainerStarted","Data":"2d5f47313d0b0f5b24347356b8258b2e4fb563505b0931f7142288b7ab7031ce"} Jul 06 22:21:16.663619 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.663592 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8hnkk" event={"ID":"edfcc999-de23-48fe-8d7d-72fb9836aa2b","Type":"ContainerStarted","Data":"67434afd92482fd5bedceb4b0785d50e1fbf8788d524fcb17660adfd1ba13cb1"} Jul 06 22:21:16.665527 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.665507 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" event={"ID":"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683","Type":"ContainerStarted","Data":"3a1e164c8cb35d42a00b5d6dba349c6ca3b5a6d669c86716690bdab201f0b00f"} Jul 06 22:21:16.665638 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.665536 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" event={"ID":"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683","Type":"ContainerStarted","Data":"220c634cee2175d996d061f71847a4d58a34f195bf1dc9a6c42bc7dfda53342f"} Jul 06 22:21:16.665638 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.665549 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" event={"ID":"5e1f4546-c7c0-4ad9-9cb1-b094b22e4683","Type":"ContainerStarted","Data":"9ddfff13efd3bf5f84264c258a3a670b05942784a33d931cf2015684ecf8d650"} Jul 06 22:21:16.666589 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.666570 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wmpgs" event={"ID":"d1befd3e-57bc-4d94-9563-ef9479a3d10c","Type":"ContainerStarted","Data":"63e09019ba5ea55b4e7c8addc60e26732dd977d6ec04591999691477292262b0"} Jul 06 22:21:16.667629 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.667609 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xl9br" event={"ID":"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d","Type":"ContainerStarted","Data":"ce69f79188f0a9b62357a84d35b2f1ce38d66cfaf6b7e30d6039da1c2b72e73a"} Jul 06 22:21:16.682543 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.682493 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75f7fdcc4-mvfbd" podStartSLOduration=1.6376438210000002 podStartE2EDuration="5.682475772s" totalImagesPullingTime="4.044831951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:21:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:21:16.681719349 +0000 UTC m=+66.783447012" watchObservedRunningTime="2026-07-06 22:21:16.682475772 +0000 UTC m=+66.784203435" Jul 06 22:21:16.710452 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:16.710407 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f8b554777-vzlm5" podStartSLOduration=1.670626946 podStartE2EDuration="5.710391636s" totalImagesPullingTime="4.03976469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:21:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:21:16.709425263 +0000 UTC m=+66.811152924" watchObservedRunningTime="2026-07-06 22:21:16.710391636 +0000 UTC m=+66.812119298" Jul 06 22:21:17.061263 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:17.061219 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:21:17.063993 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:17.063963 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/02417ea4-0e61-4d6e-9423-487aa1f2a053-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9vz26\" (UID: \"02417ea4-0e61-4d6e-9423-487aa1f2a053\") " pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:21:17.302819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:17.302784 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dhrwd\"" Jul 06 22:21:17.310009 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:17.309979 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9vz26" Jul 06 22:21:17.490134 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:17.490074 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9vz26"] Jul 06 22:21:17.495274 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:21:17.495229 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02417ea4_0e61_4d6e_9423_487aa1f2a053.slice/crio-6418cde478f834cd335fd00e14bef2803c024fbfc633f90e7592efee2d5c141b WatchSource:0}: Error finding container 6418cde478f834cd335fd00e14bef2803c024fbfc633f90e7592efee2d5c141b: Status 404 returned error can't find the container with id 6418cde478f834cd335fd00e14bef2803c024fbfc633f90e7592efee2d5c141b Jul 06 22:21:17.676857 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:17.676768 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9vz26" event={"ID":"02417ea4-0e61-4d6e-9423-487aa1f2a053","Type":"ContainerStarted","Data":"6418cde478f834cd335fd00e14bef2803c024fbfc633f90e7592efee2d5c141b"} Jul 06 22:21:18.687830 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:18.687792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xl9br" event={"ID":"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d","Type":"ContainerStarted","Data":"2d898a1c04b10a02904f851792cbb799b21f54cd19f3a7a0accea9231d478309"} Jul 06 22:21:18.688325 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:18.687863 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xl9br" event={"ID":"e1d8204a-b0f6-4ee1-b0ba-acc52c873c5d","Type":"ContainerStarted","Data":"70ca37c3ab93d595a097ad6d5fe6c26a362b68b009c7ac8f1efd58d5ec231ab5"} Jul 06 22:21:24.712074 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.712032 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8hnkk" event={"ID":"edfcc999-de23-48fe-8d7d-72fb9836aa2b","Type":"ContainerStarted","Data":"51e2c506b90dcdf238f46673c43549b232bf4ed4d54bf6d253533ff3dac65236"} Jul 06 22:21:24.712606 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.712168 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:21:24.713836 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.713807 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wmpgs" event={"ID":"d1befd3e-57bc-4d94-9563-ef9479a3d10c","Type":"ContainerStarted","Data":"9330e49ad9c4a6a0e74fe63d5b19407281f368db3381e1dde96f4074b3104423"} Jul 06 22:21:24.715471 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.715438 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9vz26" event={"ID":"02417ea4-0e61-4d6e-9423-487aa1f2a053","Type":"ContainerStarted","Data":"3d463961bd3fc51eba1e694907c001f732dcc8172645b4c83b44c684daa84bb1"} Jul 06 22:21:24.752584 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.752532 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xl9br" podStartSLOduration=73.455554082 podStartE2EDuration="1m14.752517581s" totalImagesPullingTime="1.296963499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:21:18.733130322 +0000 UTC m=+68.834857983" watchObservedRunningTime="2026-07-06 22:21:24.752517581 +0000 UTC m=+74.854245316" Jul 06 22:21:24.753084 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.753052 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8hnkk" podStartSLOduration=67.170109035 podStartE2EDuration="1m14.753044359s" totalImagesPullingTime="7.582935324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:21:24.75153766 +0000 UTC m=+74.853265322" watchObservedRunningTime="2026-07-06 22:21:24.753044359 +0000 UTC m=+74.854772051" Jul 06 22:21:24.787095 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.787039 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wmpgs" podStartSLOduration=67.023656466 podStartE2EDuration="1m14.787015243s" totalImagesPullingTime="7.763358777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:21:24.786280661 +0000 UTC m=+74.888008321" watchObservedRunningTime="2026-07-06 22:21:24.787015243 +0000 UTC m=+74.888742908" Jul 06 22:21:24.985837 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.985761 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lzdb4"] Jul 06 22:21:24.992619 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.992590 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:24.995252 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.995228 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qgwgf\"" Jul 06 22:21:24.995573 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.995553 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 22:21:24.996676 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.996528 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 22:21:24.996676 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.996612 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 22:21:24.996676 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.996646 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 22:21:24.996999 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.996977 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 22:21:24.997913 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:24.997881 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 22:21:25.124998 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.124964 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-wtmp\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.124998 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.125000 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-textfile\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.125235 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.125023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-tls\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.125235 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.125082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-accelerators-collector-config\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.125235 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.125129 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjqsg\" (UniqueName: \"kubernetes.io/projected/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-kube-api-access-qjqsg\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.125235 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.125193 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-sys\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.125235 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.125211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.125235 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.125237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-metrics-client-ca\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.125435 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.125256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-root\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226215 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-sys\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226298 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-sys\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226335 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-metrics-client-ca\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226366 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-root\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226616 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-wtmp\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226616 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-textfile\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226616 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226465 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-tls\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226616 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226536 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-root\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226616 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226585 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-accelerators-collector-config\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226616 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226594 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-wtmp\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226907 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjqsg\" (UniqueName: \"kubernetes.io/projected/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-kube-api-access-qjqsg\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.226907 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226795 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-textfile\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.227036 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.226993 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-metrics-client-ca\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.227094 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.227035 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-accelerators-collector-config\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.228687 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.228655 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.228790 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.228734 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-node-exporter-tls\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.237827 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.237776 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjqsg\" (UniqueName: \"kubernetes.io/projected/c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96-kube-api-access-qjqsg\") pod \"node-exporter-lzdb4\" (UID: \"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96\") " pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.303921 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.303887 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lzdb4" Jul 06 22:21:25.315518 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:21:25.315484 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5f6e4ee_9c5d_49cd_a8f4_f8c3dcf4eb96.slice/crio-81a0c854175fd117a045442ac4dbf460a6fd125617c5774facc864cba62081fa WatchSource:0}: Error finding container 81a0c854175fd117a045442ac4dbf460a6fd125617c5774facc864cba62081fa: Status 404 returned error can't find the container with id 81a0c854175fd117a045442ac4dbf460a6fd125617c5774facc864cba62081fa Jul 06 22:21:25.726429 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.726343 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9vz26" event={"ID":"02417ea4-0e61-4d6e-9423-487aa1f2a053","Type":"ContainerStarted","Data":"e3c77a458ceb2b955b54023808b733b5f447d3dac9c81268ae5c909a33eb55a0"} Jul 06 22:21:25.727832 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:25.727802 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lzdb4" event={"ID":"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96","Type":"ContainerStarted","Data":"81a0c854175fd117a045442ac4dbf460a6fd125617c5774facc864cba62081fa"} Jul 06 22:21:27.737579 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:27.737541 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9vz26" event={"ID":"02417ea4-0e61-4d6e-9423-487aa1f2a053","Type":"ContainerStarted","Data":"ad64d399b69064636511e0a7cb43a8cb44bda322a04231929120ec063cb30412"} Jul 06 22:21:27.739620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:27.739569 2578 generic.go:356] "Generic (PLEG): container finished" podID="c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96" containerID="8d199bb745e1f02a126ce309c8dd005dc30579040c259ed5b12ba0403349d9fd" exitCode=0 Jul 06 22:21:27.739786 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:27.739631 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lzdb4" event={"ID":"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96","Type":"ContainerDied","Data":"8d199bb745e1f02a126ce309c8dd005dc30579040c259ed5b12ba0403349d9fd"} Jul 06 22:21:27.847401 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:27.847346 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9vz26" podStartSLOduration=32.929612041 podStartE2EDuration="42.847325141s" totalImagesPullingTime="9.9177131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:20:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:21:27.845849615 +0000 UTC m=+77.947577278" watchObservedRunningTime="2026-07-06 22:21:27.847325141 +0000 UTC m=+77.949052805" Jul 06 22:21:28.748185 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:28.748146 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lzdb4" event={"ID":"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96","Type":"ContainerStarted","Data":"5f2f796dfd9491b8e26f0342b80fbda239c31765f87780600645835f25f9bf34"} Jul 06 22:21:28.748580 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:28.748231 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lzdb4" event={"ID":"c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96","Type":"ContainerStarted","Data":"1a46373f8cb6d19aa8c3e659a38ee95d00884f0ab9a3586bae7aa935951bf4f2"} Jul 06 22:21:28.772577 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:28.772527 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lzdb4" podStartSLOduration=2.488680802 podStartE2EDuration="4.772512371s" totalImagesPullingTime="2.221822156s" totalInitContainerRuntime="62.009413ms" isStatefulPod=true podCreationTimestamp="2026-07-06 22:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:21:28.771945175 +0000 UTC m=+78.873672839" watchObservedRunningTime="2026-07-06 22:21:28.772512371 +0000 UTC m=+78.874240088" Jul 06 22:21:55.730438 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:21:55.730408 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8hnkk" Jul 06 22:23:26.877334 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:26.877300 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz"] Jul 06 22:23:26.880416 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:26.880398 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:26.882925 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:26.882905 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 22:23:26.883706 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:26.883689 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 22:23:26.883819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:26.883711 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s5pch\"" Jul 06 22:23:26.889879 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:26.889856 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz"] Jul 06 22:23:26.986179 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:26.986141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:26.986360 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:26.986189 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lhvh\" (UniqueName: \"kubernetes.io/projected/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-kube-api-access-5lhvh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:26.986360 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:26.986257 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:27.087485 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:27.087432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:27.087485 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:27.087497 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhvh\" (UniqueName: \"kubernetes.io/projected/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-kube-api-access-5lhvh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:27.087663 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:27.087529 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:27.087968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:27.087944 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:27.088048 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:27.087968 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:27.099153 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:27.099105 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lhvh\" (UniqueName: \"kubernetes.io/projected/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-kube-api-access-5lhvh\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:27.189413 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:27.189303 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:27.321541 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:27.321501 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz"] Jul 06 22:23:27.325220 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:23:27.325192 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e90913_98f4_4cf9_80a9_fd8b650e7e2e.slice/crio-3804e7d5785d003554492dc54bd535ca56adaf550d8ab9ec30d5aa222b699cfe WatchSource:0}: Error finding container 3804e7d5785d003554492dc54bd535ca56adaf550d8ab9ec30d5aa222b699cfe: Status 404 returned error can't find the container with id 3804e7d5785d003554492dc54bd535ca56adaf550d8ab9ec30d5aa222b699cfe Jul 06 22:23:28.136104 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:28.136069 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" event={"ID":"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e","Type":"ContainerStarted","Data":"3804e7d5785d003554492dc54bd535ca56adaf550d8ab9ec30d5aa222b699cfe"} Jul 06 22:23:36.164500 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:36.164468 2578 generic.go:356] "Generic (PLEG): container finished" podID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerID="799bdb2c35d830956b4615aaad57cd14dc88df17a51a44ef1f04c8cd69e2730f" exitCode=0 Jul 06 22:23:36.164882 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:36.164516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" event={"ID":"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e","Type":"ContainerDied","Data":"799bdb2c35d830956b4615aaad57cd14dc88df17a51a44ef1f04c8cd69e2730f"} Jul 06 22:23:38.173413 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:38.173373 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" event={"ID":"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e","Type":"ContainerStarted","Data":"3797aa9116cdbbac02c3104bac0bbb73cf04fb091b21f4f301f882590049605a"} Jul 06 22:23:39.179967 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:39.179935 2578 generic.go:356] "Generic (PLEG): container finished" podID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerID="3797aa9116cdbbac02c3104bac0bbb73cf04fb091b21f4f301f882590049605a" exitCode=0 Jul 06 22:23:39.180453 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:39.179998 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" event={"ID":"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e","Type":"ContainerDied","Data":"3797aa9116cdbbac02c3104bac0bbb73cf04fb091b21f4f301f882590049605a"} Jul 06 22:23:48.215779 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:48.215736 2578 generic.go:356] "Generic (PLEG): container finished" podID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerID="bd2259bc36598d14e334f3af2822c3b23e62d2d0914c9d9004d5abb5edfc95ad" exitCode=0 Jul 06 22:23:48.216178 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:48.215801 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" event={"ID":"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e","Type":"ContainerDied","Data":"bd2259bc36598d14e334f3af2822c3b23e62d2d0914c9d9004d5abb5edfc95ad"} Jul 06 22:23:49.364201 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.364166 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:49.464459 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.464418 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lhvh\" (UniqueName: \"kubernetes.io/projected/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-kube-api-access-5lhvh\") pod \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " Jul 06 22:23:49.464673 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.464502 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-bundle\") pod \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " Jul 06 22:23:49.464673 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.464529 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-util\") pod \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\" (UID: \"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e\") " Jul 06 22:23:49.465164 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.465138 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-bundle" (OuterVolumeSpecName: "bundle") pod "f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" (UID: "f7e90913-98f4-4cf9-80a9-fd8b650e7e2e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:23:49.466715 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.466690 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-kube-api-access-5lhvh" (OuterVolumeSpecName: "kube-api-access-5lhvh") pod "f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" (UID: "f7e90913-98f4-4cf9-80a9-fd8b650e7e2e"). InnerVolumeSpecName "kube-api-access-5lhvh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:23:49.469244 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.469223 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-util" (OuterVolumeSpecName: "util") pod "f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" (UID: "f7e90913-98f4-4cf9-80a9-fd8b650e7e2e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:23:49.565265 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.565225 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-util\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:23:49.565265 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.565258 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lhvh\" (UniqueName: \"kubernetes.io/projected/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-kube-api-access-5lhvh\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:23:49.565265 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:49.565269 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7e90913-98f4-4cf9-80a9-fd8b650e7e2e-bundle\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:23:50.224329 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:50.224304 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" Jul 06 22:23:50.224484 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:50.224303 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pr5wz" event={"ID":"f7e90913-98f4-4cf9-80a9-fd8b650e7e2e","Type":"ContainerDied","Data":"3804e7d5785d003554492dc54bd535ca56adaf550d8ab9ec30d5aa222b699cfe"} Jul 06 22:23:50.224484 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:50.224423 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3804e7d5785d003554492dc54bd535ca56adaf550d8ab9ec30d5aa222b699cfe" Jul 06 22:23:54.695716 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.695681 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn"] Jul 06 22:23:54.696159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.695915 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerName="pull" Jul 06 22:23:54.696159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.695927 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerName="pull" Jul 06 22:23:54.696159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.695946 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerName="util" Jul 06 22:23:54.696159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.695952 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerName="util" Jul 06 22:23:54.696159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.695965 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerName="extract" Jul 06 22:23:54.696159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.695971 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerName="extract" Jul 06 22:23:54.696159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.696011 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e90913-98f4-4cf9-80a9-fd8b650e7e2e" containerName="extract" Jul 06 22:23:54.700703 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.700684 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" Jul 06 22:23:54.702940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.702920 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:23:54.703031 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.702918 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 22:23:54.703031 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.702972 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-hsk46\"" Jul 06 22:23:54.710565 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.710537 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn"] Jul 06 22:23:54.801231 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.801187 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd7799c3-2515-4dbe-b2d6-4d0311cdb19e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-452nn\" (UID: \"fd7799c3-2515-4dbe-b2d6-4d0311cdb19e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" Jul 06 22:23:54.801402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.801239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqgfj\" (UniqueName: \"kubernetes.io/projected/fd7799c3-2515-4dbe-b2d6-4d0311cdb19e-kube-api-access-zqgfj\") pod \"cert-manager-operator-controller-manager-55c444bf86-452nn\" (UID: \"fd7799c3-2515-4dbe-b2d6-4d0311cdb19e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" Jul 06 22:23:54.901648 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.901617 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd7799c3-2515-4dbe-b2d6-4d0311cdb19e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-452nn\" (UID: \"fd7799c3-2515-4dbe-b2d6-4d0311cdb19e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" Jul 06 22:23:54.901751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.901663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqgfj\" (UniqueName: \"kubernetes.io/projected/fd7799c3-2515-4dbe-b2d6-4d0311cdb19e-kube-api-access-zqgfj\") pod \"cert-manager-operator-controller-manager-55c444bf86-452nn\" (UID: \"fd7799c3-2515-4dbe-b2d6-4d0311cdb19e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" Jul 06 22:23:54.902024 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.902005 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd7799c3-2515-4dbe-b2d6-4d0311cdb19e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-452nn\" (UID: \"fd7799c3-2515-4dbe-b2d6-4d0311cdb19e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" Jul 06 22:23:54.911638 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:54.911605 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqgfj\" (UniqueName: \"kubernetes.io/projected/fd7799c3-2515-4dbe-b2d6-4d0311cdb19e-kube-api-access-zqgfj\") pod \"cert-manager-operator-controller-manager-55c444bf86-452nn\" (UID: \"fd7799c3-2515-4dbe-b2d6-4d0311cdb19e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" Jul 06 22:23:55.019245 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:55.019161 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" Jul 06 22:23:55.158866 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:55.158829 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn"] Jul 06 22:23:55.162155 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:23:55.162128 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd7799c3_2515_4dbe_b2d6_4d0311cdb19e.slice/crio-3ce910f5ce5190a459115977f3d2b9f45c194d2fc674921dbd94339fc463c694 WatchSource:0}: Error finding container 3ce910f5ce5190a459115977f3d2b9f45c194d2fc674921dbd94339fc463c694: Status 404 returned error can't find the container with id 3ce910f5ce5190a459115977f3d2b9f45c194d2fc674921dbd94339fc463c694 Jul 06 22:23:55.242925 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:55.242887 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" event={"ID":"fd7799c3-2515-4dbe-b2d6-4d0311cdb19e","Type":"ContainerStarted","Data":"3ce910f5ce5190a459115977f3d2b9f45c194d2fc674921dbd94339fc463c694"} Jul 06 22:23:57.251645 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:57.251614 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" event={"ID":"fd7799c3-2515-4dbe-b2d6-4d0311cdb19e","Type":"ContainerStarted","Data":"7dea6d94dac3f3c0aeec706d304a544207c78dba871222441dfff11016d64085"} Jul 06 22:23:57.277805 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:57.277703 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-452nn" podStartSLOduration=1.705955856 podStartE2EDuration="3.27768708s" totalImagesPullingTime="1.571731224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:23:57.275585168 +0000 UTC m=+227.377312830" watchObservedRunningTime="2026-07-06 22:23:57.27768708 +0000 UTC m=+227.379414742" Jul 06 22:23:58.315619 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.315587 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f"] Jul 06 22:23:58.318950 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.318933 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.321490 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.321472 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 22:23:58.322169 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.322154 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s5pch\"" Jul 06 22:23:58.322169 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.322161 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 22:23:58.328145 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.328106 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f"] Jul 06 22:23:58.427917 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.427878 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.428090 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.427931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnxr5\" (UniqueName: \"kubernetes.io/projected/e7dcd640-5dc9-47de-8f94-3f23a9543466-kube-api-access-hnxr5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.428090 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.427999 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.528840 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.528805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnxr5\" (UniqueName: \"kubernetes.io/projected/e7dcd640-5dc9-47de-8f94-3f23a9543466-kube-api-access-hnxr5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.529034 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.528849 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.529034 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.528964 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.529382 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.529353 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.529516 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.529477 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.540810 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.540780 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnxr5\" (UniqueName: \"kubernetes.io/projected/e7dcd640-5dc9-47de-8f94-3f23a9543466-kube-api-access-hnxr5\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.628324 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.628286 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:23:58.759219 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.759182 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f"] Jul 06 22:23:58.762866 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:23:58.762839 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7dcd640_5dc9_47de_8f94_3f23a9543466.slice/crio-c12f0d497a33ad23b4d62bc8bc1ad0dd7fefc0f03f916ba7390b99a84fb56c7f WatchSource:0}: Error finding container c12f0d497a33ad23b4d62bc8bc1ad0dd7fefc0f03f916ba7390b99a84fb56c7f: Status 404 returned error can't find the container with id c12f0d497a33ad23b4d62bc8bc1ad0dd7fefc0f03f916ba7390b99a84fb56c7f Jul 06 22:23:58.798608 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.798575 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j2sbb"] Jul 06 22:23:58.803200 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.803178 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:23:58.805614 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.805592 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-t9slt\"" Jul 06 22:23:58.805614 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.805604 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 22:23:58.805778 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.805605 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 22:23:58.811927 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.811906 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j2sbb"] Jul 06 22:23:58.932596 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.932507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zb65\" (UniqueName: \"kubernetes.io/projected/8d3e7b91-8268-4517-a656-a3fb5881da33-kube-api-access-4zb65\") pod \"cert-manager-webhook-6486ffd997-j2sbb\" (UID: \"8d3e7b91-8268-4517-a656-a3fb5881da33\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:23:58.932596 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:58.932546 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3e7b91-8268-4517-a656-a3fb5881da33-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j2sbb\" (UID: \"8d3e7b91-8268-4517-a656-a3fb5881da33\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:23:59.033786 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.033748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zb65\" (UniqueName: \"kubernetes.io/projected/8d3e7b91-8268-4517-a656-a3fb5881da33-kube-api-access-4zb65\") pod \"cert-manager-webhook-6486ffd997-j2sbb\" (UID: \"8d3e7b91-8268-4517-a656-a3fb5881da33\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:23:59.033933 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.033795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3e7b91-8268-4517-a656-a3fb5881da33-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j2sbb\" (UID: \"8d3e7b91-8268-4517-a656-a3fb5881da33\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:23:59.045241 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.045207 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d3e7b91-8268-4517-a656-a3fb5881da33-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-j2sbb\" (UID: \"8d3e7b91-8268-4517-a656-a3fb5881da33\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:23:59.045422 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.045402 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zb65\" (UniqueName: \"kubernetes.io/projected/8d3e7b91-8268-4517-a656-a3fb5881da33-kube-api-access-4zb65\") pod \"cert-manager-webhook-6486ffd997-j2sbb\" (UID: \"8d3e7b91-8268-4517-a656-a3fb5881da33\") " pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:23:59.112916 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.112882 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:23:59.239554 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.239519 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-j2sbb"] Jul 06 22:23:59.244518 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:23:59.243564 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d3e7b91_8268_4517_a656_a3fb5881da33.slice/crio-c07f9a89b6230de0e3936670a9d02c23238ca143146c829d986f2da273355fad WatchSource:0}: Error finding container c07f9a89b6230de0e3936670a9d02c23238ca143146c829d986f2da273355fad: Status 404 returned error can't find the container with id c07f9a89b6230de0e3936670a9d02c23238ca143146c829d986f2da273355fad Jul 06 22:23:59.259961 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.259929 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" event={"ID":"8d3e7b91-8268-4517-a656-a3fb5881da33","Type":"ContainerStarted","Data":"c07f9a89b6230de0e3936670a9d02c23238ca143146c829d986f2da273355fad"} Jul 06 22:23:59.261336 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.261312 2578 generic.go:356] "Generic (PLEG): container finished" podID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerID="4eca93496aca345f080f79d9d36bceba6be8bb3cf25510842c7070f95ba81aa9" exitCode=0 Jul 06 22:23:59.261444 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.261355 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" event={"ID":"e7dcd640-5dc9-47de-8f94-3f23a9543466","Type":"ContainerDied","Data":"4eca93496aca345f080f79d9d36bceba6be8bb3cf25510842c7070f95ba81aa9"} Jul 06 22:23:59.261444 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:23:59.261386 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" event={"ID":"e7dcd640-5dc9-47de-8f94-3f23a9543466","Type":"ContainerStarted","Data":"c12f0d497a33ad23b4d62bc8bc1ad0dd7fefc0f03f916ba7390b99a84fb56c7f"} Jul 06 22:24:01.271944 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:01.271905 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" event={"ID":"e7dcd640-5dc9-47de-8f94-3f23a9543466","Type":"ContainerStarted","Data":"28f9d73bd843d66646c655496f0f24864e824c115f53998e7e0cd17eed556b63"} Jul 06 22:24:02.278099 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:02.278060 2578 generic.go:356] "Generic (PLEG): container finished" podID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerID="28f9d73bd843d66646c655496f0f24864e824c115f53998e7e0cd17eed556b63" exitCode=0 Jul 06 22:24:02.278533 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:02.278140 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" event={"ID":"e7dcd640-5dc9-47de-8f94-3f23a9543466","Type":"ContainerDied","Data":"28f9d73bd843d66646c655496f0f24864e824c115f53998e7e0cd17eed556b63"} Jul 06 22:24:03.283327 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:03.283288 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" event={"ID":"8d3e7b91-8268-4517-a656-a3fb5881da33","Type":"ContainerStarted","Data":"76407acc04923de3ed56c3d3a43b0064640603f5c68829fe7cb2992dd1174562"} Jul 06 22:24:03.283804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:03.283360 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:24:03.285427 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:03.285400 2578 generic.go:356] "Generic (PLEG): container finished" podID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerID="6241493ab5cf07a1493d1ca644f710d197ae6a0a49e7bcb09bff656863aea359" exitCode=0 Jul 06 22:24:03.285523 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:03.285484 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" event={"ID":"e7dcd640-5dc9-47de-8f94-3f23a9543466","Type":"ContainerDied","Data":"6241493ab5cf07a1493d1ca644f710d197ae6a0a49e7bcb09bff656863aea359"} Jul 06 22:24:03.303406 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:03.303359 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" podStartSLOduration=1.9161597609999999 podStartE2EDuration="5.303340518s" totalImagesPullingTime="3.387180757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:03.302791378 +0000 UTC m=+233.404519040" watchObservedRunningTime="2026-07-06 22:24:03.303340518 +0000 UTC m=+233.405068182" Jul 06 22:24:04.410922 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.410898 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:24:04.471666 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.471625 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnxr5\" (UniqueName: \"kubernetes.io/projected/e7dcd640-5dc9-47de-8f94-3f23a9543466-kube-api-access-hnxr5\") pod \"e7dcd640-5dc9-47de-8f94-3f23a9543466\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " Jul 06 22:24:04.471666 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.471670 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-bundle\") pod \"e7dcd640-5dc9-47de-8f94-3f23a9543466\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " Jul 06 22:24:04.471911 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.471690 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-util\") pod \"e7dcd640-5dc9-47de-8f94-3f23a9543466\" (UID: \"e7dcd640-5dc9-47de-8f94-3f23a9543466\") " Jul 06 22:24:04.472142 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.472095 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-bundle" (OuterVolumeSpecName: "bundle") pod "e7dcd640-5dc9-47de-8f94-3f23a9543466" (UID: "e7dcd640-5dc9-47de-8f94-3f23a9543466"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:24:04.473741 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.473714 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7dcd640-5dc9-47de-8f94-3f23a9543466-kube-api-access-hnxr5" (OuterVolumeSpecName: "kube-api-access-hnxr5") pod "e7dcd640-5dc9-47de-8f94-3f23a9543466" (UID: "e7dcd640-5dc9-47de-8f94-3f23a9543466"). InnerVolumeSpecName "kube-api-access-hnxr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:24:04.476531 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.476492 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-util" (OuterVolumeSpecName: "util") pod "e7dcd640-5dc9-47de-8f94-3f23a9543466" (UID: "e7dcd640-5dc9-47de-8f94-3f23a9543466"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:24:04.572810 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.572732 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-bundle\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:24:04.572810 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.572758 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7dcd640-5dc9-47de-8f94-3f23a9543466-util\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:24:04.572810 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:04.572767 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnxr5\" (UniqueName: \"kubernetes.io/projected/e7dcd640-5dc9-47de-8f94-3f23a9543466-kube-api-access-hnxr5\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:24:05.295288 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:05.295258 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" Jul 06 22:24:05.295288 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:05.295269 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f85q5f" event={"ID":"e7dcd640-5dc9-47de-8f94-3f23a9543466","Type":"ContainerDied","Data":"c12f0d497a33ad23b4d62bc8bc1ad0dd7fefc0f03f916ba7390b99a84fb56c7f"} Jul 06 22:24:05.295502 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:05.295325 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c12f0d497a33ad23b4d62bc8bc1ad0dd7fefc0f03f916ba7390b99a84fb56c7f" Jul 06 22:24:09.292197 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:09.292161 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-j2sbb" Jul 06 22:24:18.379400 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.379372 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2"] Jul 06 22:24:18.379794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.379615 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerName="util" Jul 06 22:24:18.379794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.379625 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerName="util" Jul 06 22:24:18.379794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.379636 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerName="pull" Jul 06 22:24:18.379794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.379641 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerName="pull" Jul 06 22:24:18.379794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.379646 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerName="extract" Jul 06 22:24:18.379794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.379651 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerName="extract" Jul 06 22:24:18.379794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.379701 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7dcd640-5dc9-47de-8f94-3f23a9543466" containerName="extract" Jul 06 22:24:18.435298 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.435262 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2"] Jul 06 22:24:18.435461 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.435388 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.437807 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.437787 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 22:24:18.438484 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.438470 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-s5pch\"" Jul 06 22:24:18.438549 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.438493 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 22:24:18.467491 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.467453 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.467683 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.467612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85vvp\" (UniqueName: \"kubernetes.io/projected/175943c5-d03f-4519-9396-14f014df45a9-kube-api-access-85vvp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.467683 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.467643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.568948 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.568912 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85vvp\" (UniqueName: \"kubernetes.io/projected/175943c5-d03f-4519-9396-14f014df45a9-kube-api-access-85vvp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.569152 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.568951 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.569152 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.568983 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.569317 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.569296 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.569369 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.569328 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.580964 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.580941 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85vvp\" (UniqueName: \"kubernetes.io/projected/175943c5-d03f-4519-9396-14f014df45a9-kube-api-access-85vvp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.744805 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.744725 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:18.899256 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:18.899102 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2"] Jul 06 22:24:18.901809 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:24:18.901780 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175943c5_d03f_4519_9396_14f014df45a9.slice/crio-61a2b8f54a2d476bbbe681c004765893412240c76f951dd3b885508da5758959 WatchSource:0}: Error finding container 61a2b8f54a2d476bbbe681c004765893412240c76f951dd3b885508da5758959: Status 404 returned error can't find the container with id 61a2b8f54a2d476bbbe681c004765893412240c76f951dd3b885508da5758959 Jul 06 22:24:19.046782 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.046744 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j9mlb"] Jul 06 22:24:19.067149 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.067104 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j9mlb"] Jul 06 22:24:19.067304 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.067196 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j9mlb" Jul 06 22:24:19.069639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.069620 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-d75pl\"" Jul 06 22:24:19.173956 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.173912 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfz6l\" (UniqueName: \"kubernetes.io/projected/74bc4b0a-4122-4412-aaa0-e580f7635fdf-kube-api-access-hfz6l\") pod \"cert-manager-55b4d954d5-j9mlb\" (UID: \"74bc4b0a-4122-4412-aaa0-e580f7635fdf\") " pod="cert-manager/cert-manager-55b4d954d5-j9mlb" Jul 06 22:24:19.174176 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.173974 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74bc4b0a-4122-4412-aaa0-e580f7635fdf-bound-sa-token\") pod \"cert-manager-55b4d954d5-j9mlb\" (UID: \"74bc4b0a-4122-4412-aaa0-e580f7635fdf\") " pod="cert-manager/cert-manager-55b4d954d5-j9mlb" Jul 06 22:24:19.274713 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.274612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74bc4b0a-4122-4412-aaa0-e580f7635fdf-bound-sa-token\") pod \"cert-manager-55b4d954d5-j9mlb\" (UID: \"74bc4b0a-4122-4412-aaa0-e580f7635fdf\") " pod="cert-manager/cert-manager-55b4d954d5-j9mlb" Jul 06 22:24:19.274713 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.274685 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfz6l\" (UniqueName: \"kubernetes.io/projected/74bc4b0a-4122-4412-aaa0-e580f7635fdf-kube-api-access-hfz6l\") pod \"cert-manager-55b4d954d5-j9mlb\" (UID: \"74bc4b0a-4122-4412-aaa0-e580f7635fdf\") " pod="cert-manager/cert-manager-55b4d954d5-j9mlb" Jul 06 22:24:19.285476 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.285442 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74bc4b0a-4122-4412-aaa0-e580f7635fdf-bound-sa-token\") pod \"cert-manager-55b4d954d5-j9mlb\" (UID: \"74bc4b0a-4122-4412-aaa0-e580f7635fdf\") " pod="cert-manager/cert-manager-55b4d954d5-j9mlb" Jul 06 22:24:19.285639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.285614 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfz6l\" (UniqueName: \"kubernetes.io/projected/74bc4b0a-4122-4412-aaa0-e580f7635fdf-kube-api-access-hfz6l\") pod \"cert-manager-55b4d954d5-j9mlb\" (UID: \"74bc4b0a-4122-4412-aaa0-e580f7635fdf\") " pod="cert-manager/cert-manager-55b4d954d5-j9mlb" Jul 06 22:24:19.356153 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.356099 2578 generic.go:356] "Generic (PLEG): container finished" podID="175943c5-d03f-4519-9396-14f014df45a9" containerID="a068f7ad4079ac96d5760d07c96f67f59f7b050ee97758b71a5c4d4379fe6893" exitCode=0 Jul 06 22:24:19.356153 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.356145 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" event={"ID":"175943c5-d03f-4519-9396-14f014df45a9","Type":"ContainerDied","Data":"a068f7ad4079ac96d5760d07c96f67f59f7b050ee97758b71a5c4d4379fe6893"} Jul 06 22:24:19.356365 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.356197 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" event={"ID":"175943c5-d03f-4519-9396-14f014df45a9","Type":"ContainerStarted","Data":"61a2b8f54a2d476bbbe681c004765893412240c76f951dd3b885508da5758959"} Jul 06 22:24:19.423139 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.423079 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j9mlb" Jul 06 22:24:19.550862 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:19.550776 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j9mlb"] Jul 06 22:24:19.553308 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:24:19.553282 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74bc4b0a_4122_4412_aaa0_e580f7635fdf.slice/crio-41d42bcfdab636a165931e099100f45c24857825c3473a4c03dec3b1960bf465 WatchSource:0}: Error finding container 41d42bcfdab636a165931e099100f45c24857825c3473a4c03dec3b1960bf465: Status 404 returned error can't find the container with id 41d42bcfdab636a165931e099100f45c24857825c3473a4c03dec3b1960bf465 Jul 06 22:24:20.361794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:20.361758 2578 generic.go:356] "Generic (PLEG): container finished" podID="175943c5-d03f-4519-9396-14f014df45a9" containerID="fa5993fbe54077a4809975dccbb32bdd3082b2e99cebd407d531e18d54b24268" exitCode=0 Jul 06 22:24:20.361979 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:20.361831 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" event={"ID":"175943c5-d03f-4519-9396-14f014df45a9","Type":"ContainerDied","Data":"fa5993fbe54077a4809975dccbb32bdd3082b2e99cebd407d531e18d54b24268"} Jul 06 22:24:20.363338 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:20.363310 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j9mlb" event={"ID":"74bc4b0a-4122-4412-aaa0-e580f7635fdf","Type":"ContainerStarted","Data":"189e3ded117608ed6a3cdc39edd51b83f7c6824a51b151fd6a238709c69856f9"} Jul 06 22:24:20.363492 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:20.363349 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j9mlb" event={"ID":"74bc4b0a-4122-4412-aaa0-e580f7635fdf","Type":"ContainerStarted","Data":"41d42bcfdab636a165931e099100f45c24857825c3473a4c03dec3b1960bf465"} Jul 06 22:24:20.403221 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:20.403175 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-j9mlb" podStartSLOduration=1.403161025 podStartE2EDuration="1.403161025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:20.402273125 +0000 UTC m=+250.504000788" watchObservedRunningTime="2026-07-06 22:24:20.403161025 +0000 UTC m=+250.504888687" Jul 06 22:24:21.369449 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:21.369411 2578 generic.go:356] "Generic (PLEG): container finished" podID="175943c5-d03f-4519-9396-14f014df45a9" containerID="2cf86f1652793de3b34be6ed1e8a5812873722f308b33fad02bf4dd3fa184fec" exitCode=0 Jul 06 22:24:21.369934 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:21.369497 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" event={"ID":"175943c5-d03f-4519-9396-14f014df45a9","Type":"ContainerDied","Data":"2cf86f1652793de3b34be6ed1e8a5812873722f308b33fad02bf4dd3fa184fec"} Jul 06 22:24:22.494358 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.494337 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:22.601660 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.601622 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85vvp\" (UniqueName: \"kubernetes.io/projected/175943c5-d03f-4519-9396-14f014df45a9-kube-api-access-85vvp\") pod \"175943c5-d03f-4519-9396-14f014df45a9\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " Jul 06 22:24:22.601660 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.601665 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-bundle\") pod \"175943c5-d03f-4519-9396-14f014df45a9\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " Jul 06 22:24:22.601905 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.601718 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-util\") pod \"175943c5-d03f-4519-9396-14f014df45a9\" (UID: \"175943c5-d03f-4519-9396-14f014df45a9\") " Jul 06 22:24:22.602359 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.602332 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-bundle" (OuterVolumeSpecName: "bundle") pod "175943c5-d03f-4519-9396-14f014df45a9" (UID: "175943c5-d03f-4519-9396-14f014df45a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:24:22.603688 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.603661 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/175943c5-d03f-4519-9396-14f014df45a9-kube-api-access-85vvp" (OuterVolumeSpecName: "kube-api-access-85vvp") pod "175943c5-d03f-4519-9396-14f014df45a9" (UID: "175943c5-d03f-4519-9396-14f014df45a9"). InnerVolumeSpecName "kube-api-access-85vvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:24:22.607644 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.607614 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-util" (OuterVolumeSpecName: "util") pod "175943c5-d03f-4519-9396-14f014df45a9" (UID: "175943c5-d03f-4519-9396-14f014df45a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:24:22.703072 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.702988 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-util\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:24:22.703072 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.703016 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85vvp\" (UniqueName: \"kubernetes.io/projected/175943c5-d03f-4519-9396-14f014df45a9-kube-api-access-85vvp\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:24:22.703072 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:22.703027 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/175943c5-d03f-4519-9396-14f014df45a9-bundle\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:24:23.379423 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:23.379393 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" Jul 06 22:24:23.379423 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:23.379403 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afqdvw2" event={"ID":"175943c5-d03f-4519-9396-14f014df45a9","Type":"ContainerDied","Data":"61a2b8f54a2d476bbbe681c004765893412240c76f951dd3b885508da5758959"} Jul 06 22:24:23.379623 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:23.379442 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61a2b8f54a2d476bbbe681c004765893412240c76f951dd3b885508da5758959" Jul 06 22:24:35.668530 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.668445 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm"] Jul 06 22:24:35.668992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.668801 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175943c5-d03f-4519-9396-14f014df45a9" containerName="util" Jul 06 22:24:35.668992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.668818 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="175943c5-d03f-4519-9396-14f014df45a9" containerName="util" Jul 06 22:24:35.668992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.668831 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175943c5-d03f-4519-9396-14f014df45a9" containerName="pull" Jul 06 22:24:35.668992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.668839 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="175943c5-d03f-4519-9396-14f014df45a9" containerName="pull" Jul 06 22:24:35.668992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.668849 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175943c5-d03f-4519-9396-14f014df45a9" containerName="extract" Jul 06 22:24:35.668992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.668857 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="175943c5-d03f-4519-9396-14f014df45a9" containerName="extract" Jul 06 22:24:35.668992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.668918 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="175943c5-d03f-4519-9396-14f014df45a9" containerName="extract" Jul 06 22:24:35.671937 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.671913 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.674931 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.674910 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 22:24:35.675035 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.674943 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 22:24:35.675183 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.675168 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-th8c4\"" Jul 06 22:24:35.675271 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.675250 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 22:24:35.675348 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.675335 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 22:24:35.693033 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.692995 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm"] Jul 06 22:24:35.798567 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.798533 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fbc927a3-37e6-41ee-bb97-9f1283fff37c-manifests\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.798759 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.798571 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fbc927a3-37e6-41ee-bb97-9f1283fff37c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.798759 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.798601 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbc927a3-37e6-41ee-bb97-9f1283fff37c-webhook-cert\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.798759 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.798628 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssdr9\" (UniqueName: \"kubernetes.io/projected/fbc927a3-37e6-41ee-bb97-9f1283fff37c-kube-api-access-ssdr9\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.798759 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.798675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbc927a3-37e6-41ee-bb97-9f1283fff37c-tmp\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.899886 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.899851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fbc927a3-37e6-41ee-bb97-9f1283fff37c-manifests\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.899886 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.899888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fbc927a3-37e6-41ee-bb97-9f1283fff37c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.900166 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.899916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbc927a3-37e6-41ee-bb97-9f1283fff37c-webhook-cert\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.900166 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.899944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssdr9\" (UniqueName: \"kubernetes.io/projected/fbc927a3-37e6-41ee-bb97-9f1283fff37c-kube-api-access-ssdr9\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.900166 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.899980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbc927a3-37e6-41ee-bb97-9f1283fff37c-tmp\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.900333 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.900277 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fbc927a3-37e6-41ee-bb97-9f1283fff37c-manifests\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.902251 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.902230 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbc927a3-37e6-41ee-bb97-9f1283fff37c-tmp\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.902498 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.902476 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbc927a3-37e6-41ee-bb97-9f1283fff37c-webhook-cert\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.902558 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.902541 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fbc927a3-37e6-41ee-bb97-9f1283fff37c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.909848 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.909813 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssdr9\" (UniqueName: \"kubernetes.io/projected/fbc927a3-37e6-41ee-bb97-9f1283fff37c-kube-api-access-ssdr9\") pod \"opendatahub-operator-controller-manager-c6597d756-5qfcm\" (UID: \"fbc927a3-37e6-41ee-bb97-9f1283fff37c\") " pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:35.981934 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:35.981849 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:36.112530 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:36.112505 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm"] Jul 06 22:24:36.115147 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:24:36.115101 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc927a3_37e6_41ee_bb97_9f1283fff37c.slice/crio-c4c8df870b96e2e8588774e231966e9b4923c5af6defe0930d3d771908c87583 WatchSource:0}: Error finding container c4c8df870b96e2e8588774e231966e9b4923c5af6defe0930d3d771908c87583: Status 404 returned error can't find the container with id c4c8df870b96e2e8588774e231966e9b4923c5af6defe0930d3d771908c87583 Jul 06 22:24:36.432624 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:36.432595 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" event={"ID":"fbc927a3-37e6-41ee-bb97-9f1283fff37c","Type":"ContainerStarted","Data":"c4c8df870b96e2e8588774e231966e9b4923c5af6defe0930d3d771908c87583"} Jul 06 22:24:39.449382 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:39.449340 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" event={"ID":"fbc927a3-37e6-41ee-bb97-9f1283fff37c","Type":"ContainerStarted","Data":"69739fa46e920d4511d90d1b4dfea33f648908b80c551a3f7d09ba1aaa9f8488"} Jul 06 22:24:40.454393 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:40.454306 2578 generic.go:356] "Generic (PLEG): container finished" podID="fbc927a3-37e6-41ee-bb97-9f1283fff37c" containerID="69739fa46e920d4511d90d1b4dfea33f648908b80c551a3f7d09ba1aaa9f8488" exitCode=0 Jul 06 22:24:40.454934 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:40.454392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" event={"ID":"fbc927a3-37e6-41ee-bb97-9f1283fff37c","Type":"ContainerDied","Data":"69739fa46e920d4511d90d1b4dfea33f648908b80c551a3f7d09ba1aaa9f8488"} Jul 06 22:24:41.460500 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:41.460466 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" event={"ID":"fbc927a3-37e6-41ee-bb97-9f1283fff37c","Type":"ContainerStarted","Data":"30079841c64be8f49d009ebb9c8e4a7c2227a5c92111a8953cbf4d7ec5e619ef"} Jul 06 22:24:41.460964 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:41.460538 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:41.492533 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:41.492455 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" podStartSLOduration=2.147407261 podStartE2EDuration="6.492432334s" totalImagesPullingTime="3.265745323s" totalInitContainerRuntime="1.07927975s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:41.489698971 +0000 UTC m=+271.591426634" watchObservedRunningTime="2026-07-06 22:24:41.492432334 +0000 UTC m=+271.594159999" Jul 06 22:24:52.467283 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:52.467249 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c6597d756-5qfcm" Jul 06 22:24:55.353888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.353849 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7588b845d6-lq682"] Jul 06 22:24:55.368052 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.368028 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.371129 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.371091 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 22:24:55.371467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.371358 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 22:24:55.371467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.371414 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 22:24:55.371668 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.371643 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 22:24:55.376787 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.376765 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7588b845d6-lq682"] Jul 06 22:24:55.447138 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.447073 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nl2k4"] Jul 06 22:24:55.450477 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.450454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:55.452813 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.452787 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 22:24:55.452923 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.452854 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vw8bw\"" Jul 06 22:24:55.460896 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.460842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32aefe50-699f-4ed3-9ce5-576d68543567-tls-certs\") pod \"kube-auth-proxy-7588b845d6-lq682\" (UID: \"32aefe50-699f-4ed3-9ce5-576d68543567\") " pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.460896 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.460879 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32aefe50-699f-4ed3-9ce5-576d68543567-tmp\") pod \"kube-auth-proxy-7588b845d6-lq682\" (UID: \"32aefe50-699f-4ed3-9ce5-576d68543567\") " pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.461079 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.460909 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t87vz\" (UniqueName: \"kubernetes.io/projected/32aefe50-699f-4ed3-9ce5-576d68543567-kube-api-access-t87vz\") pod \"kube-auth-proxy-7588b845d6-lq682\" (UID: \"32aefe50-699f-4ed3-9ce5-576d68543567\") " pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.463605 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.463584 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nl2k4"] Jul 06 22:24:55.561920 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.561891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-622xj\" (UniqueName: \"kubernetes.io/projected/025fc94b-f743-42ef-a475-6adee1c6eaf3-kube-api-access-622xj\") pod \"odh-model-controller-7998c67dc4-nl2k4\" (UID: \"025fc94b-f743-42ef-a475-6adee1c6eaf3\") " pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:55.561920 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.561929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/025fc94b-f743-42ef-a475-6adee1c6eaf3-cert\") pod \"odh-model-controller-7998c67dc4-nl2k4\" (UID: \"025fc94b-f743-42ef-a475-6adee1c6eaf3\") " pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:55.562258 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.561956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32aefe50-699f-4ed3-9ce5-576d68543567-tls-certs\") pod \"kube-auth-proxy-7588b845d6-lq682\" (UID: \"32aefe50-699f-4ed3-9ce5-576d68543567\") " pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.562258 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.562007 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32aefe50-699f-4ed3-9ce5-576d68543567-tmp\") pod \"kube-auth-proxy-7588b845d6-lq682\" (UID: \"32aefe50-699f-4ed3-9ce5-576d68543567\") " pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.562258 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.562046 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t87vz\" (UniqueName: \"kubernetes.io/projected/32aefe50-699f-4ed3-9ce5-576d68543567-kube-api-access-t87vz\") pod \"kube-auth-proxy-7588b845d6-lq682\" (UID: \"32aefe50-699f-4ed3-9ce5-576d68543567\") " pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.564440 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.564419 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32aefe50-699f-4ed3-9ce5-576d68543567-tls-certs\") pod \"kube-auth-proxy-7588b845d6-lq682\" (UID: \"32aefe50-699f-4ed3-9ce5-576d68543567\") " pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.564604 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.564589 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32aefe50-699f-4ed3-9ce5-576d68543567-tmp\") pod \"kube-auth-proxy-7588b845d6-lq682\" (UID: \"32aefe50-699f-4ed3-9ce5-576d68543567\") " pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.571417 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.571388 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t87vz\" (UniqueName: \"kubernetes.io/projected/32aefe50-699f-4ed3-9ce5-576d68543567-kube-api-access-t87vz\") pod \"kube-auth-proxy-7588b845d6-lq682\" (UID: \"32aefe50-699f-4ed3-9ce5-576d68543567\") " pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.663194 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.663074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-622xj\" (UniqueName: \"kubernetes.io/projected/025fc94b-f743-42ef-a475-6adee1c6eaf3-kube-api-access-622xj\") pod \"odh-model-controller-7998c67dc4-nl2k4\" (UID: \"025fc94b-f743-42ef-a475-6adee1c6eaf3\") " pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:55.663194 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.663181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/025fc94b-f743-42ef-a475-6adee1c6eaf3-cert\") pod \"odh-model-controller-7998c67dc4-nl2k4\" (UID: \"025fc94b-f743-42ef-a475-6adee1c6eaf3\") " pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:55.663441 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:24:55.663377 2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 22:24:55.663483 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:24:55.663456 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/025fc94b-f743-42ef-a475-6adee1c6eaf3-cert podName:025fc94b-f743-42ef-a475-6adee1c6eaf3 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:56.16343388 +0000 UTC m=+286.265161529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/025fc94b-f743-42ef-a475-6adee1c6eaf3-cert") pod "odh-model-controller-7998c67dc4-nl2k4" (UID: "025fc94b-f743-42ef-a475-6adee1c6eaf3") : secret "odh-model-controller-webhook-cert" not found Jul 06 22:24:55.676640 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.676598 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-622xj\" (UniqueName: \"kubernetes.io/projected/025fc94b-f743-42ef-a475-6adee1c6eaf3-kube-api-access-622xj\") pod \"odh-model-controller-7998c67dc4-nl2k4\" (UID: \"025fc94b-f743-42ef-a475-6adee1c6eaf3\") " pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:55.678156 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.678136 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" Jul 06 22:24:55.828134 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:55.828095 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7588b845d6-lq682"] Jul 06 22:24:55.829738 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:24:55.829708 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32aefe50_699f_4ed3_9ce5_576d68543567.slice/crio-cf6479cdfaef1de21337155e8a8e86e46d3887b0f40e95d6135fcb4cf57babf1 WatchSource:0}: Error finding container cf6479cdfaef1de21337155e8a8e86e46d3887b0f40e95d6135fcb4cf57babf1: Status 404 returned error can't find the container with id cf6479cdfaef1de21337155e8a8e86e46d3887b0f40e95d6135fcb4cf57babf1 Jul 06 22:24:56.167654 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.167617 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/025fc94b-f743-42ef-a475-6adee1c6eaf3-cert\") pod \"odh-model-controller-7998c67dc4-nl2k4\" (UID: \"025fc94b-f743-42ef-a475-6adee1c6eaf3\") " pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:56.167877 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:24:56.167798 2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 22:24:56.167877 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:24:56.167874 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/025fc94b-f743-42ef-a475-6adee1c6eaf3-cert podName:025fc94b-f743-42ef-a475-6adee1c6eaf3 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:57.167854265 +0000 UTC m=+287.269581907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/025fc94b-f743-42ef-a475-6adee1c6eaf3-cert") pod "odh-model-controller-7998c67dc4-nl2k4" (UID: "025fc94b-f743-42ef-a475-6adee1c6eaf3") : secret "odh-model-controller-webhook-cert" not found Jul 06 22:24:56.456785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.456699 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gd744"] Jul 06 22:24:56.461393 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.461372 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:24:56.465104 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.465079 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 22:24:56.465267 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.465101 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jm26r\"" Jul 06 22:24:56.474126 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.474075 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gd744"] Jul 06 22:24:56.527218 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.527154 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" event={"ID":"32aefe50-699f-4ed3-9ce5-576d68543567","Type":"ContainerStarted","Data":"cf6479cdfaef1de21337155e8a8e86e46d3887b0f40e95d6135fcb4cf57babf1"} Jul 06 22:24:56.570637 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.570338 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p"] Jul 06 22:24:56.571314 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.571273 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07eb0e43-b3b8-4a30-9337-ab887649d05f-tls-certs\") pod \"model-serving-api-54d89bd5fc-gd744\" (UID: \"07eb0e43-b3b8-4a30-9337-ab887649d05f\") " pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:24:56.571447 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.571394 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69z5b\" (UniqueName: \"kubernetes.io/projected/07eb0e43-b3b8-4a30-9337-ab887649d05f-kube-api-access-69z5b\") pod \"model-serving-api-54d89bd5fc-gd744\" (UID: \"07eb0e43-b3b8-4a30-9337-ab887649d05f\") " pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:24:56.574314 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.574291 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.578719 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.577517 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6x958\"" Jul 06 22:24:56.578719 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.577872 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 22:24:56.578719 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.578052 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 22:24:56.578719 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.578266 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 22:24:56.578719 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.578508 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 22:24:56.587138 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.585876 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p"] Jul 06 22:24:56.672752 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.672710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-istio-token\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.672951 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.672809 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69z5b\" (UniqueName: \"kubernetes.io/projected/07eb0e43-b3b8-4a30-9337-ab887649d05f-kube-api-access-69z5b\") pod \"model-serving-api-54d89bd5fc-gd744\" (UID: \"07eb0e43-b3b8-4a30-9337-ab887649d05f\") " pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:24:56.672951 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.672923 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.673066 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.672981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07eb0e43-b3b8-4a30-9337-ab887649d05f-tls-certs\") pod \"model-serving-api-54d89bd5fc-gd744\" (UID: \"07eb0e43-b3b8-4a30-9337-ab887649d05f\") " pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:24:56.673066 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.673014 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.673066 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.673047 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-cacerts\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.673302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.673077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj226\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-kube-api-access-kj226\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.673302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.673103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37670cb-cd7a-4022-8f65-106315ec016e-local-certs\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.673302 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:24:56.673152 2578 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 06 22:24:56.673302 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:24:56.673218 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07eb0e43-b3b8-4a30-9337-ab887649d05f-tls-certs podName:07eb0e43-b3b8-4a30-9337-ab887649d05f nodeName:}" failed. No retries permitted until 2026-07-06 22:24:57.173197644 +0000 UTC m=+287.274925296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/07eb0e43-b3b8-4a30-9337-ab887649d05f-tls-certs") pod "model-serving-api-54d89bd5fc-gd744" (UID: "07eb0e43-b3b8-4a30-9337-ab887649d05f") : secret "model-serving-api-tls" not found Jul 06 22:24:56.673302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.673259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.685343 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.685314 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69z5b\" (UniqueName: \"kubernetes.io/projected/07eb0e43-b3b8-4a30-9337-ab887649d05f-kube-api-access-69z5b\") pod \"model-serving-api-54d89bd5fc-gd744\" (UID: \"07eb0e43-b3b8-4a30-9337-ab887649d05f\") " pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:24:56.774639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.774295 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.774639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.774371 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-istio-token\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.774639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.774417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.774639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.774471 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.774639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.774500 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-cacerts\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.774639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.774527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj226\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-kube-api-access-kj226\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.774639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.774551 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37670cb-cd7a-4022-8f65-106315ec016e-local-certs\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.775414 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.775378 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.777912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.777858 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-cacerts\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.778457 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.778411 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37670cb-cd7a-4022-8f65-106315ec016e-local-certs\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.778807 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.778782 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.780582 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.780535 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.785612 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.785588 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj226\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-kube-api-access-kj226\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.786034 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.786014 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-istio-token\") pod \"istiod-openshift-gateway-7574459656-4qz9p\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:56.888821 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:56.888657 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:24:57.041333 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.041248 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p"] Jul 06 22:24:57.044071 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:24:57.044040 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda37670cb_cd7a_4022_8f65_106315ec016e.slice/crio-2584db13c15e0c49f70380fb874c113ad26174e19341ab87b3a8b3bc7130fa78 WatchSource:0}: Error finding container 2584db13c15e0c49f70380fb874c113ad26174e19341ab87b3a8b3bc7130fa78: Status 404 returned error can't find the container with id 2584db13c15e0c49f70380fb874c113ad26174e19341ab87b3a8b3bc7130fa78 Jul 06 22:24:57.178334 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.178302 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07eb0e43-b3b8-4a30-9337-ab887649d05f-tls-certs\") pod \"model-serving-api-54d89bd5fc-gd744\" (UID: \"07eb0e43-b3b8-4a30-9337-ab887649d05f\") " pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:24:57.178539 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.178391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/025fc94b-f743-42ef-a475-6adee1c6eaf3-cert\") pod \"odh-model-controller-7998c67dc4-nl2k4\" (UID: \"025fc94b-f743-42ef-a475-6adee1c6eaf3\") " pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:57.181601 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.181568 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/025fc94b-f743-42ef-a475-6adee1c6eaf3-cert\") pod \"odh-model-controller-7998c67dc4-nl2k4\" (UID: \"025fc94b-f743-42ef-a475-6adee1c6eaf3\") " pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:57.181737 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.181568 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07eb0e43-b3b8-4a30-9337-ab887649d05f-tls-certs\") pod \"model-serving-api-54d89bd5fc-gd744\" (UID: \"07eb0e43-b3b8-4a30-9337-ab887649d05f\") " pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:24:57.263645 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.263604 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:24:57.376044 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.375813 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:24:57.422037 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.422002 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nl2k4"] Jul 06 22:24:57.425665 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:24:57.425636 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025fc94b_f743_42ef_a475_6adee1c6eaf3.slice/crio-772f740f503a6922dbd1b4b793aab922a19eb65ca876ed14672592a0c787faea WatchSource:0}: Error finding container 772f740f503a6922dbd1b4b793aab922a19eb65ca876ed14672592a0c787faea: Status 404 returned error can't find the container with id 772f740f503a6922dbd1b4b793aab922a19eb65ca876ed14672592a0c787faea Jul 06 22:24:57.533389 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.533356 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" event={"ID":"a37670cb-cd7a-4022-8f65-106315ec016e","Type":"ContainerStarted","Data":"2584db13c15e0c49f70380fb874c113ad26174e19341ab87b3a8b3bc7130fa78"} Jul 06 22:24:57.534426 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.534405 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" event={"ID":"025fc94b-f743-42ef-a475-6adee1c6eaf3","Type":"ContainerStarted","Data":"772f740f503a6922dbd1b4b793aab922a19eb65ca876ed14672592a0c787faea"} Jul 06 22:24:57.539085 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:57.539058 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gd744"] Jul 06 22:24:57.541789 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:24:57.541763 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07eb0e43_b3b8_4a30_9337_ab887649d05f.slice/crio-78ef910534c3a567467e73498dd6621aa1099c2e03e47ee099a97d1302e978de WatchSource:0}: Error finding container 78ef910534c3a567467e73498dd6621aa1099c2e03e47ee099a97d1302e978de: Status 404 returned error can't find the container with id 78ef910534c3a567467e73498dd6621aa1099c2e03e47ee099a97d1302e978de Jul 06 22:24:58.543036 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:24:58.542993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gd744" event={"ID":"07eb0e43-b3b8-4a30-9337-ab887649d05f","Type":"ContainerStarted","Data":"78ef910534c3a567467e73498dd6621aa1099c2e03e47ee099a97d1302e978de"} Jul 06 22:25:01.955374 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:01.955324 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:25:01.955821 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:01.955420 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:25:02.566398 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.566360 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" event={"ID":"32aefe50-699f-4ed3-9ce5-576d68543567","Type":"ContainerStarted","Data":"dc3d72224359fbf0d4b6131ad7e6f1f27e870b44bd809be0c58781fd1a6d2302"} Jul 06 22:25:02.568787 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.568751 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" event={"ID":"a37670cb-cd7a-4022-8f65-106315ec016e","Type":"ContainerStarted","Data":"e52c38a23694f206caaa29225df4ef3f02e1af560751d930c938fcc106e4f85d"} Jul 06 22:25:02.568938 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.568843 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:25:02.571576 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.571544 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" event={"ID":"025fc94b-f743-42ef-a475-6adee1c6eaf3","Type":"ContainerStarted","Data":"01ca0b52c5b3630aa0d5c6671abdb00dca10f62b9690e4a5147fbadde56716ac"} Jul 06 22:25:02.571857 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.571835 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:25:02.573508 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.573480 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gd744" event={"ID":"07eb0e43-b3b8-4a30-9337-ab887649d05f","Type":"ContainerStarted","Data":"4962694d98096fe3a18a9d9135e36fac78b957ab8ba7e0a0ca9b7a3a8ff95754"} Jul 06 22:25:02.573794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.573766 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:25:02.584449 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.584386 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7588b845d6-lq682" podStartSLOduration=1.460828696 podStartE2EDuration="7.584365373s" totalImagesPullingTime="6.123536677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:02.583298537 +0000 UTC m=+292.685026200" watchObservedRunningTime="2026-07-06 22:25:02.584365373 +0000 UTC m=+292.686093035" Jul 06 22:25:02.603720 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.603669 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" podStartSLOduration=1.694696344 podStartE2EDuration="6.603648152s" totalImagesPullingTime="4.908951808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:02.602203954 +0000 UTC m=+292.703931629" watchObservedRunningTime="2026-07-06 22:25:02.603648152 +0000 UTC m=+292.705375815" Jul 06 22:25:02.625688 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.625630 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" podStartSLOduration=3.030949903 podStartE2EDuration="7.625611847s" totalImagesPullingTime="4.594661944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:02.624337817 +0000 UTC m=+292.726065480" watchObservedRunningTime="2026-07-06 22:25:02.625611847 +0000 UTC m=+292.727339508" Jul 06 22:25:02.642933 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:02.642874 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gd744" podStartSLOduration=2.2313896 podStartE2EDuration="6.642860034s" totalImagesPullingTime="4.411470434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:02.64199343 +0000 UTC m=+292.743721099" watchObservedRunningTime="2026-07-06 22:25:02.642860034 +0000 UTC m=+292.744587696" Jul 06 22:25:03.580075 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:03.580046 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:25:07.129991 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.129953 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mglnv"] Jul 06 22:25:07.133538 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.133516 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:07.135782 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.135749 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 22:25:07.135782 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.135773 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-99wbq\"" Jul 06 22:25:07.143250 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.143229 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz"] Jul 06 22:25:07.146715 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.146694 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:07.146910 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.146885 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mglnv"] Jul 06 22:25:07.149005 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.148981 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 22:25:07.149092 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.149046 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-w8bfv\"" Jul 06 22:25:07.156503 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.156477 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz"] Jul 06 22:25:07.264043 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.264008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9037b453-a672-4ba4-9a9b-845a773ade2c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rdznz\" (UID: \"9037b453-a672-4ba4-9a9b-845a773ade2c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:07.264235 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.264053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mglnv\" (UID: \"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:07.264235 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.264192 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl9mc\" (UniqueName: \"kubernetes.io/projected/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-kube-api-access-zl9mc\") pod \"kserve-controller-manager-d6f4b65b8-mglnv\" (UID: \"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:07.264305 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.264240 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89vhs\" (UniqueName: \"kubernetes.io/projected/9037b453-a672-4ba4-9a9b-845a773ade2c-kube-api-access-89vhs\") pod \"llmisvc-controller-manager-777c6bd7d5-rdznz\" (UID: \"9037b453-a672-4ba4-9a9b-845a773ade2c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:07.365281 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.365252 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mglnv\" (UID: \"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:07.365476 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.365314 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl9mc\" (UniqueName: \"kubernetes.io/projected/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-kube-api-access-zl9mc\") pod \"kserve-controller-manager-d6f4b65b8-mglnv\" (UID: \"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:07.365476 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.365345 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89vhs\" (UniqueName: \"kubernetes.io/projected/9037b453-a672-4ba4-9a9b-845a773ade2c-kube-api-access-89vhs\") pod \"llmisvc-controller-manager-777c6bd7d5-rdznz\" (UID: \"9037b453-a672-4ba4-9a9b-845a773ade2c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:07.365476 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.365388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9037b453-a672-4ba4-9a9b-845a773ade2c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rdznz\" (UID: \"9037b453-a672-4ba4-9a9b-845a773ade2c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:07.365476 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:25:07.365429 2578 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 06 22:25:07.365661 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:25:07.365494 2578 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 06 22:25:07.365661 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:25:07.365504 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-cert podName:31538d2a-93f9-4dee-bccf-6aec3dfa0e0c nodeName:}" failed. No retries permitted until 2026-07-06 22:25:07.865482721 +0000 UTC m=+297.967210367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-cert") pod "kserve-controller-manager-d6f4b65b8-mglnv" (UID: "31538d2a-93f9-4dee-bccf-6aec3dfa0e0c") : secret "kserve-webhook-server-cert" not found Jul 06 22:25:07.365661 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:25:07.365542 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9037b453-a672-4ba4-9a9b-845a773ade2c-cert podName:9037b453-a672-4ba4-9a9b-845a773ade2c nodeName:}" failed. No retries permitted until 2026-07-06 22:25:07.865526148 +0000 UTC m=+297.967253789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9037b453-a672-4ba4-9a9b-845a773ade2c-cert") pod "llmisvc-controller-manager-777c6bd7d5-rdznz" (UID: "9037b453-a672-4ba4-9a9b-845a773ade2c") : secret "llmisvc-webhook-server-cert" not found Jul 06 22:25:07.388467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.388392 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89vhs\" (UniqueName: \"kubernetes.io/projected/9037b453-a672-4ba4-9a9b-845a773ade2c-kube-api-access-89vhs\") pod \"llmisvc-controller-manager-777c6bd7d5-rdznz\" (UID: \"9037b453-a672-4ba4-9a9b-845a773ade2c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:07.399662 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.399626 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl9mc\" (UniqueName: \"kubernetes.io/projected/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-kube-api-access-zl9mc\") pod \"kserve-controller-manager-d6f4b65b8-mglnv\" (UID: \"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:07.870345 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.870301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9037b453-a672-4ba4-9a9b-845a773ade2c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rdznz\" (UID: \"9037b453-a672-4ba4-9a9b-845a773ade2c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:07.870530 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.870368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mglnv\" (UID: \"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:07.870530 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:25:07.870518 2578 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 06 22:25:07.870618 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:25:07.870583 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-cert podName:31538d2a-93f9-4dee-bccf-6aec3dfa0e0c nodeName:}" failed. No retries permitted until 2026-07-06 22:25:08.870567637 +0000 UTC m=+298.972295276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-cert") pod "kserve-controller-manager-d6f4b65b8-mglnv" (UID: "31538d2a-93f9-4dee-bccf-6aec3dfa0e0c") : secret "kserve-webhook-server-cert" not found Jul 06 22:25:07.872804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:07.872778 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9037b453-a672-4ba4-9a9b-845a773ade2c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rdznz\" (UID: \"9037b453-a672-4ba4-9a9b-845a773ade2c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:08.059415 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:08.059379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:08.192494 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:08.192469 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz"] Jul 06 22:25:08.194809 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:25:08.194780 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9037b453_a672_4ba4_9a9b_845a773ade2c.slice/crio-5a9d97926c13c3bbf7486af329a7739981662c864deb1504d201f6db6910fa21 WatchSource:0}: Error finding container 5a9d97926c13c3bbf7486af329a7739981662c864deb1504d201f6db6910fa21: Status 404 returned error can't find the container with id 5a9d97926c13c3bbf7486af329a7739981662c864deb1504d201f6db6910fa21 Jul 06 22:25:08.600223 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:08.600189 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" event={"ID":"9037b453-a672-4ba4-9a9b-845a773ade2c","Type":"ContainerStarted","Data":"5a9d97926c13c3bbf7486af329a7739981662c864deb1504d201f6db6910fa21"} Jul 06 22:25:08.880260 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:08.880179 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mglnv\" (UID: \"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:08.882922 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:08.882882 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/31538d2a-93f9-4dee-bccf-6aec3dfa0e0c-cert\") pod \"kserve-controller-manager-d6f4b65b8-mglnv\" (UID: \"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:08.946627 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:08.946594 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:09.279782 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.279701 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p"] Jul 06 22:25:09.281254 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.280589 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" podUID="a37670cb-cd7a-4022-8f65-106315ec016e" containerName="discovery" containerID="cri-o://e52c38a23694f206caaa29225df4ef3f02e1af560751d930c938fcc106e4f85d" gracePeriod=30 Jul 06 22:25:09.372890 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.372861 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mglnv"] Jul 06 22:25:09.417554 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:25:09.417519 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31538d2a_93f9_4dee_bccf_6aec3dfa0e0c.slice/crio-39ad8e1fe252e421a33cc0fe8ca1a9077c97d3b3129d11f40830c715acd7b2c3 WatchSource:0}: Error finding container 39ad8e1fe252e421a33cc0fe8ca1a9077c97d3b3129d11f40830c715acd7b2c3: Status 404 returned error can't find the container with id 39ad8e1fe252e421a33cc0fe8ca1a9077c97d3b3129d11f40830c715acd7b2c3 Jul 06 22:25:09.608240 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.608156 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" event={"ID":"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c","Type":"ContainerStarted","Data":"39ad8e1fe252e421a33cc0fe8ca1a9077c97d3b3129d11f40830c715acd7b2c3"} Jul 06 22:25:09.610043 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.609901 2578 generic.go:356] "Generic (PLEG): container finished" podID="a37670cb-cd7a-4022-8f65-106315ec016e" containerID="e52c38a23694f206caaa29225df4ef3f02e1af560751d930c938fcc106e4f85d" exitCode=0 Jul 06 22:25:09.610043 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.609956 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" event={"ID":"a37670cb-cd7a-4022-8f65-106315ec016e","Type":"ContainerDied","Data":"e52c38a23694f206caaa29225df4ef3f02e1af560751d930c938fcc106e4f85d"} Jul 06 22:25:09.610043 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.609993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" event={"ID":"a37670cb-cd7a-4022-8f65-106315ec016e","Type":"ContainerDied","Data":"2584db13c15e0c49f70380fb874c113ad26174e19341ab87b3a8b3bc7130fa78"} Jul 06 22:25:09.610043 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.610015 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2584db13c15e0c49f70380fb874c113ad26174e19341ab87b3a8b3bc7130fa78" Jul 06 22:25:09.620880 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.620857 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:25:09.787576 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.787478 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-cacerts\") pod \"a37670cb-cd7a-4022-8f65-106315ec016e\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " Jul 06 22:25:09.787576 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.787516 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-istio-token\") pod \"a37670cb-cd7a-4022-8f65-106315ec016e\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " Jul 06 22:25:09.787576 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.787551 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-dns-cert\") pod \"a37670cb-cd7a-4022-8f65-106315ec016e\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " Jul 06 22:25:09.787844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.787587 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37670cb-cd7a-4022-8f65-106315ec016e-local-certs\") pod \"a37670cb-cd7a-4022-8f65-106315ec016e\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " Jul 06 22:25:09.787844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.787627 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-kubeconfig\") pod \"a37670cb-cd7a-4022-8f65-106315ec016e\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " Jul 06 22:25:09.787844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.787662 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj226\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-kube-api-access-kj226\") pod \"a37670cb-cd7a-4022-8f65-106315ec016e\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " Jul 06 22:25:09.787844 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.787716 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-ca-configmap\") pod \"a37670cb-cd7a-4022-8f65-106315ec016e\" (UID: \"a37670cb-cd7a-4022-8f65-106315ec016e\") " Jul 06 22:25:09.788453 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.788401 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "a37670cb-cd7a-4022-8f65-106315ec016e" (UID: "a37670cb-cd7a-4022-8f65-106315ec016e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:25:09.790385 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.790339 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-cacerts" (OuterVolumeSpecName: "cacerts") pod "a37670cb-cd7a-4022-8f65-106315ec016e" (UID: "a37670cb-cd7a-4022-8f65-106315ec016e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:25:09.790492 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.790379 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-kube-api-access-kj226" (OuterVolumeSpecName: "kube-api-access-kj226") pod "a37670cb-cd7a-4022-8f65-106315ec016e" (UID: "a37670cb-cd7a-4022-8f65-106315ec016e"). InnerVolumeSpecName "kube-api-access-kj226". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:25:09.790492 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.790392 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "a37670cb-cd7a-4022-8f65-106315ec016e" (UID: "a37670cb-cd7a-4022-8f65-106315ec016e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:25:09.790591 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.790515 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a37670cb-cd7a-4022-8f65-106315ec016e" (UID: "a37670cb-cd7a-4022-8f65-106315ec016e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:25:09.790731 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.790712 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "a37670cb-cd7a-4022-8f65-106315ec016e" (UID: "a37670cb-cd7a-4022-8f65-106315ec016e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:25:09.791031 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.791005 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37670cb-cd7a-4022-8f65-106315ec016e-local-certs" (OuterVolumeSpecName: "local-certs") pod "a37670cb-cd7a-4022-8f65-106315ec016e" (UID: "a37670cb-cd7a-4022-8f65-106315ec016e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:25:09.889225 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.889176 2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-ca-configmap\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:25:09.889225 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.889214 2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-cacerts\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:25:09.889225 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.889228 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:25:09.889495 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.889244 2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-csr-dns-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:25:09.889495 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.889256 2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37670cb-cd7a-4022-8f65-106315ec016e-local-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:25:09.889495 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.889269 2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a37670cb-cd7a-4022-8f65-106315ec016e-istio-kubeconfig\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:25:09.889495 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:09.889281 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj226\" (UniqueName: \"kubernetes.io/projected/a37670cb-cd7a-4022-8f65-106315ec016e-kube-api-access-kj226\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:25:10.268039 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:10.268016 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 22:25:10.614930 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:10.614903 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" Jul 06 22:25:10.637912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:10.637876 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p"] Jul 06 22:25:10.645625 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:10.645593 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p"] Jul 06 22:25:11.620755 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:11.620717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" event={"ID":"9037b453-a672-4ba4-9a9b-845a773ade2c","Type":"ContainerStarted","Data":"60bddc5fb198211efd48203d070f59719dc26424af8119d37333d82130c31bd6"} Jul 06 22:25:11.621165 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:11.620836 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:11.644862 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:11.644799 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" podStartSLOduration=1.33813875 podStartE2EDuration="4.644781786s" totalImagesPullingTime="3.306643036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:11.643634664 +0000 UTC m=+301.745362323" watchObservedRunningTime="2026-07-06 22:25:11.644781786 +0000 UTC m=+301.746509447" Jul 06 22:25:12.378633 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:12.378602 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a37670cb-cd7a-4022-8f65-106315ec016e" path="/var/lib/kubelet/pods/a37670cb-cd7a-4022-8f65-106315ec016e/volumes" Jul 06 22:25:12.626697 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:12.626653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" event={"ID":"31538d2a-93f9-4dee-bccf-6aec3dfa0e0c","Type":"ContainerStarted","Data":"fa6ac93db9d8017ceafab783f71b429eafb997b04ebcb208b35f0598da81123d"} Jul 06 22:25:12.627092 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:12.626889 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:12.664628 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:12.664560 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" podStartSLOduration=2.880221134 podStartE2EDuration="5.664529634s" totalImagesPullingTime="2.7843085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:12.661285032 +0000 UTC m=+302.763012708" watchObservedRunningTime="2026-07-06 22:25:12.664529634 +0000 UTC m=+302.766257300" Jul 06 22:25:13.580491 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:13.580459 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-nl2k4" Jul 06 22:25:13.582399 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:13.582374 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gd744" Jul 06 22:25:14.579212 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:14.579160 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-4qz9p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.23:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 22:25:14.579601 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:14.579236 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4qz9p" podUID="a37670cb-cd7a-4022-8f65-106315ec016e" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.23:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 22:25:42.629599 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:42.629566 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rdznz" Jul 06 22:25:43.636757 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:43.636717 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mglnv" Jul 06 22:25:56.273657 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.273625 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ttzfl"] Jul 06 22:25:56.274106 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.273910 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a37670cb-cd7a-4022-8f65-106315ec016e" containerName="discovery" Jul 06 22:25:56.274106 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.273921 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a37670cb-cd7a-4022-8f65-106315ec016e" containerName="discovery" Jul 06 22:25:56.274106 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.273976 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a37670cb-cd7a-4022-8f65-106315ec016e" containerName="discovery" Jul 06 22:25:56.279285 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.279259 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" Jul 06 22:25:56.282488 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.282464 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 22:25:56.284310 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.284285 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2sb2f\"" Jul 06 22:25:56.284438 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.284354 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 22:25:56.289532 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.289471 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ttzfl"] Jul 06 22:25:56.360379 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.360342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6kwg\" (UniqueName: \"kubernetes.io/projected/9014bacc-f071-4f74-9690-0168c03e7003-kube-api-access-h6kwg\") pod \"kuadrant-operator-catalog-ttzfl\" (UID: \"9014bacc-f071-4f74-9690-0168c03e7003\") " pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" Jul 06 22:25:56.461625 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.461591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6kwg\" (UniqueName: \"kubernetes.io/projected/9014bacc-f071-4f74-9690-0168c03e7003-kube-api-access-h6kwg\") pod \"kuadrant-operator-catalog-ttzfl\" (UID: \"9014bacc-f071-4f74-9690-0168c03e7003\") " pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" Jul 06 22:25:56.471958 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.471923 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6kwg\" (UniqueName: \"kubernetes.io/projected/9014bacc-f071-4f74-9690-0168c03e7003-kube-api-access-h6kwg\") pod \"kuadrant-operator-catalog-ttzfl\" (UID: \"9014bacc-f071-4f74-9690-0168c03e7003\") " pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" Jul 06 22:25:56.590433 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.590395 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" Jul 06 22:25:56.630874 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.630841 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ttzfl"] Jul 06 22:25:56.725485 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.725461 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ttzfl"] Jul 06 22:25:56.727373 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:25:56.727345 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9014bacc_f071_4f74_9690_0168c03e7003.slice/crio-adb0a48f108742144071304495c89ea2863165de352c0bd854019634b62e3c26 WatchSource:0}: Error finding container adb0a48f108742144071304495c89ea2863165de352c0bd854019634b62e3c26: Status 404 returned error can't find the container with id adb0a48f108742144071304495c89ea2863165de352c0bd854019634b62e3c26 Jul 06 22:25:56.728809 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.728787 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:25:56.820786 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.820750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" event={"ID":"9014bacc-f071-4f74-9690-0168c03e7003","Type":"ContainerStarted","Data":"adb0a48f108742144071304495c89ea2863165de352c0bd854019634b62e3c26"} Jul 06 22:25:56.847014 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.846933 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5dphw"] Jul 06 22:25:56.851064 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.851047 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5dphw" Jul 06 22:25:56.863128 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.863093 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5dphw"] Jul 06 22:25:56.966062 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:56.966027 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbdt6\" (UniqueName: \"kubernetes.io/projected/96ea8dad-1100-463d-bbcc-d94d07b84603-kube-api-access-sbdt6\") pod \"kuadrant-operator-catalog-5dphw\" (UID: \"96ea8dad-1100-463d-bbcc-d94d07b84603\") " pod="kuadrant-system/kuadrant-operator-catalog-5dphw" Jul 06 22:25:57.067456 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:57.067419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbdt6\" (UniqueName: \"kubernetes.io/projected/96ea8dad-1100-463d-bbcc-d94d07b84603-kube-api-access-sbdt6\") pod \"kuadrant-operator-catalog-5dphw\" (UID: \"96ea8dad-1100-463d-bbcc-d94d07b84603\") " pod="kuadrant-system/kuadrant-operator-catalog-5dphw" Jul 06 22:25:57.077643 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:57.077618 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbdt6\" (UniqueName: \"kubernetes.io/projected/96ea8dad-1100-463d-bbcc-d94d07b84603-kube-api-access-sbdt6\") pod \"kuadrant-operator-catalog-5dphw\" (UID: \"96ea8dad-1100-463d-bbcc-d94d07b84603\") " pod="kuadrant-system/kuadrant-operator-catalog-5dphw" Jul 06 22:25:57.159806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:57.159705 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5dphw" Jul 06 22:25:57.306032 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:57.306011 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5dphw"] Jul 06 22:25:57.307667 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:25:57.307646 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ea8dad_1100_463d_bbcc_d94d07b84603.slice/crio-e9100b94a48ff03e193ccb7d0ebfc546896759bf9a493e69843a7ca019efd2b7 WatchSource:0}: Error finding container e9100b94a48ff03e193ccb7d0ebfc546896759bf9a493e69843a7ca019efd2b7: Status 404 returned error can't find the container with id e9100b94a48ff03e193ccb7d0ebfc546896759bf9a493e69843a7ca019efd2b7 Jul 06 22:25:57.828412 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:57.828378 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5dphw" event={"ID":"96ea8dad-1100-463d-bbcc-d94d07b84603","Type":"ContainerStarted","Data":"e9100b94a48ff03e193ccb7d0ebfc546896759bf9a493e69843a7ca019efd2b7"} Jul 06 22:25:59.838700 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:59.838660 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" event={"ID":"9014bacc-f071-4f74-9690-0168c03e7003","Type":"ContainerStarted","Data":"dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73"} Jul 06 22:25:59.838700 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:59.838697 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" podUID="9014bacc-f071-4f74-9690-0168c03e7003" containerName="registry-server" containerID="cri-o://dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73" gracePeriod=2 Jul 06 22:25:59.840224 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:59.840197 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5dphw" event={"ID":"96ea8dad-1100-463d-bbcc-d94d07b84603","Type":"ContainerStarted","Data":"08005ba6457711be8c8b04f7c847ec60ad5c8d95e68acb0f66234303845363dd"} Jul 06 22:25:59.859980 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:59.859932 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" podStartSLOduration=1.744625125 podStartE2EDuration="3.859916732s" totalImagesPullingTime="2.115291607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:59.858935686 +0000 UTC m=+349.960663348" watchObservedRunningTime="2026-07-06 22:25:59.859916732 +0000 UTC m=+349.961644403" Jul 06 22:25:59.877443 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:25:59.877394 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5dphw" podStartSLOduration=2.341714612 podStartE2EDuration="3.877374564s" totalImagesPullingTime="1.535659952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:59.876695807 +0000 UTC m=+349.978423471" watchObservedRunningTime="2026-07-06 22:25:59.877374564 +0000 UTC m=+349.979102227" Jul 06 22:26:00.080967 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.080944 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" Jul 06 22:26:00.195379 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.195276 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6kwg\" (UniqueName: \"kubernetes.io/projected/9014bacc-f071-4f74-9690-0168c03e7003-kube-api-access-h6kwg\") pod \"9014bacc-f071-4f74-9690-0168c03e7003\" (UID: \"9014bacc-f071-4f74-9690-0168c03e7003\") " Jul 06 22:26:00.197504 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.197468 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9014bacc-f071-4f74-9690-0168c03e7003-kube-api-access-h6kwg" (OuterVolumeSpecName: "kube-api-access-h6kwg") pod "9014bacc-f071-4f74-9690-0168c03e7003" (UID: "9014bacc-f071-4f74-9690-0168c03e7003"). InnerVolumeSpecName "kube-api-access-h6kwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:26:00.296471 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.296436 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6kwg\" (UniqueName: \"kubernetes.io/projected/9014bacc-f071-4f74-9690-0168c03e7003-kube-api-access-h6kwg\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:00.846473 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.846441 2578 generic.go:356] "Generic (PLEG): container finished" podID="9014bacc-f071-4f74-9690-0168c03e7003" containerID="dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73" exitCode=0 Jul 06 22:26:00.846859 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.846481 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" Jul 06 22:26:00.846859 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.846536 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" event={"ID":"9014bacc-f071-4f74-9690-0168c03e7003","Type":"ContainerDied","Data":"dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73"} Jul 06 22:26:00.846859 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.846605 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ttzfl" event={"ID":"9014bacc-f071-4f74-9690-0168c03e7003","Type":"ContainerDied","Data":"adb0a48f108742144071304495c89ea2863165de352c0bd854019634b62e3c26"} Jul 06 22:26:00.846859 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.846625 2578 scope.go:117] "RemoveContainer" containerID="dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73" Jul 06 22:26:00.856236 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.856213 2578 scope.go:117] "RemoveContainer" containerID="dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73" Jul 06 22:26:00.856516 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:26:00.856498 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73\": container with ID starting with dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73 not found: ID does not exist" containerID="dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73" Jul 06 22:26:00.856566 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.856524 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73"} err="failed to get container status \"dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73\": rpc error: code = NotFound desc = could not find container \"dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73\": container with ID starting with dfb4c89400751c76060d770846af2b757fdcc830f1240baafd6d66e279b82e73 not found: ID does not exist" Jul 06 22:26:00.866890 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.866865 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ttzfl"] Jul 06 22:26:00.874740 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:00.874720 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ttzfl"] Jul 06 22:26:02.378587 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:02.378486 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9014bacc-f071-4f74-9690-0168c03e7003" path="/var/lib/kubelet/pods/9014bacc-f071-4f74-9690-0168c03e7003/volumes" Jul 06 22:26:07.160224 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:07.160188 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5dphw" Jul 06 22:26:07.160645 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:07.160267 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5dphw" Jul 06 22:26:07.183600 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:07.183567 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5dphw" Jul 06 22:26:07.900479 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:07.900444 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5dphw" Jul 06 22:26:11.888350 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:11.888316 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz"] Jul 06 22:26:11.888717 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:11.888603 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9014bacc-f071-4f74-9690-0168c03e7003" containerName="registry-server" Jul 06 22:26:11.888717 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:11.888615 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9014bacc-f071-4f74-9690-0168c03e7003" containerName="registry-server" Jul 06 22:26:11.888717 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:11.888668 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9014bacc-f071-4f74-9690-0168c03e7003" containerName="registry-server" Jul 06 22:26:11.893078 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:11.893056 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:11.895569 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:11.895548 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fcb48\"" Jul 06 22:26:11.900809 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:11.900663 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz"] Jul 06 22:26:12.090597 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.090558 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.090756 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.090607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.090756 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.090691 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkg2v\" (UniqueName: \"kubernetes.io/projected/0a69ad95-6583-444c-add0-abaf3f90c01a-kube-api-access-kkg2v\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.191939 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.191851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.191939 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.191895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.191939 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.191930 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkg2v\" (UniqueName: \"kubernetes.io/projected/0a69ad95-6583-444c-add0-abaf3f90c01a-kube-api-access-kkg2v\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.192246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.192232 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.192338 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.192317 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.200985 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.200957 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkg2v\" (UniqueName: \"kubernetes.io/projected/0a69ad95-6583-444c-add0-abaf3f90c01a-kube-api-access-kkg2v\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.203824 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.203803 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:12.337399 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.336055 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz"] Jul 06 22:26:12.339517 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:26:12.339491 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a69ad95_6583_444c_add0_abaf3f90c01a.slice/crio-ca7d9a116eed5c494633e30e6a20f58c580445cc1f50d92952725cd6d76ad73a WatchSource:0}: Error finding container ca7d9a116eed5c494633e30e6a20f58c580445cc1f50d92952725cd6d76ad73a: Status 404 returned error can't find the container with id ca7d9a116eed5c494633e30e6a20f58c580445cc1f50d92952725cd6d76ad73a Jul 06 22:26:12.486074 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.486035 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6"] Jul 06 22:26:12.489380 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.489360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.498946 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.498913 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6"] Jul 06 22:26:12.594698 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.594657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgv2m\" (UniqueName: \"kubernetes.io/projected/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-kube-api-access-hgv2m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.594851 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.594719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.594851 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.594774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.695957 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.695867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgv2m\" (UniqueName: \"kubernetes.io/projected/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-kube-api-access-hgv2m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.695957 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.695925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.696205 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.695975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.696357 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.696337 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.696402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.696376 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.706213 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.706187 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgv2m\" (UniqueName: \"kubernetes.io/projected/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-kube-api-access-hgv2m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.818612 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.818577 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:12.902063 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.902009 2578 generic.go:356] "Generic (PLEG): container finished" podID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerID="a1db35e5a9ecae382da610d181a6f52fb39ed4042d10453162a3542bf559ff52" exitCode=0 Jul 06 22:26:12.902675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.902106 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" event={"ID":"0a69ad95-6583-444c-add0-abaf3f90c01a","Type":"ContainerDied","Data":"a1db35e5a9ecae382da610d181a6f52fb39ed4042d10453162a3542bf559ff52"} Jul 06 22:26:12.902675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.902488 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" event={"ID":"0a69ad95-6583-444c-add0-abaf3f90c01a","Type":"ContainerStarted","Data":"ca7d9a116eed5c494633e30e6a20f58c580445cc1f50d92952725cd6d76ad73a"} Jul 06 22:26:12.953638 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:12.953614 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6"] Jul 06 22:26:12.956086 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:26:12.956056 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5142f95c_b34c_47b2_8fbf_f9ad73f39db9.slice/crio-9776e14698cb27e3dd664df7740d1c25f3d721759d2ed0f4a504364c68227fc8 WatchSource:0}: Error finding container 9776e14698cb27e3dd664df7740d1c25f3d721759d2ed0f4a504364c68227fc8: Status 404 returned error can't find the container with id 9776e14698cb27e3dd664df7740d1c25f3d721759d2ed0f4a504364c68227fc8 Jul 06 22:26:13.080663 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.080630 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs"] Jul 06 22:26:13.084031 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.084012 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.097805 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.097778 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs"] Jul 06 22:26:13.100236 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.100211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.100368 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.100247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4n58\" (UniqueName: \"kubernetes.io/projected/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-kube-api-access-v4n58\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.100368 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.100283 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.200813 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.200775 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.201007 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.200827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4n58\" (UniqueName: \"kubernetes.io/projected/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-kube-api-access-v4n58\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.201007 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.200867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.201201 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.201180 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.201276 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.201232 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.210412 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.210382 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4n58\" (UniqueName: \"kubernetes.io/projected/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-kube-api-access-v4n58\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.414467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.414430 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:13.481900 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.481862 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw"] Jul 06 22:26:13.486892 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.486868 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.495189 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.495166 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw"] Jul 06 22:26:13.503243 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.503218 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.503409 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.503259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.503409 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.503296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtgn5\" (UniqueName: \"kubernetes.io/projected/903ade62-f582-4c3f-b698-b69b9ada17b8-kube-api-access-jtgn5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.549952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.549808 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs"] Jul 06 22:26:13.552068 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:26:13.552042 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f24e8ea_1db1_47fa_ad5e_08ebe84e54d6.slice/crio-61ada3292805489280ebb2eca4f0dbce446005fc53df6454b73f7d5027740b7d WatchSource:0}: Error finding container 61ada3292805489280ebb2eca4f0dbce446005fc53df6454b73f7d5027740b7d: Status 404 returned error can't find the container with id 61ada3292805489280ebb2eca4f0dbce446005fc53df6454b73f7d5027740b7d Jul 06 22:26:13.604045 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.604012 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.604260 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.604056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.604260 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.604091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtgn5\" (UniqueName: \"kubernetes.io/projected/903ade62-f582-4c3f-b698-b69b9ada17b8-kube-api-access-jtgn5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.604453 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.604433 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.604510 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.604466 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.617265 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.617238 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtgn5\" (UniqueName: \"kubernetes.io/projected/903ade62-f582-4c3f-b698-b69b9ada17b8-kube-api-access-jtgn5\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.798017 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.797940 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:13.911651 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.911620 2578 generic.go:356] "Generic (PLEG): container finished" podID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerID="bb5ac07239f6d43d47b5d6601e674f874b7ae6e02cd8f849161e79bdf3988cd3" exitCode=0 Jul 06 22:26:13.912101 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.911698 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" event={"ID":"5142f95c-b34c-47b2-8fbf-f9ad73f39db9","Type":"ContainerDied","Data":"bb5ac07239f6d43d47b5d6601e674f874b7ae6e02cd8f849161e79bdf3988cd3"} Jul 06 22:26:13.912101 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.911749 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" event={"ID":"5142f95c-b34c-47b2-8fbf-f9ad73f39db9","Type":"ContainerStarted","Data":"9776e14698cb27e3dd664df7740d1c25f3d721759d2ed0f4a504364c68227fc8"} Jul 06 22:26:13.913419 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.913395 2578 generic.go:356] "Generic (PLEG): container finished" podID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerID="7ff5427efbaf6f4022c8c490b3515bcbbdcc823b86dc17e91ad41aff8e0d1905" exitCode=0 Jul 06 22:26:13.913547 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.913464 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" event={"ID":"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6","Type":"ContainerDied","Data":"7ff5427efbaf6f4022c8c490b3515bcbbdcc823b86dc17e91ad41aff8e0d1905"} Jul 06 22:26:13.913547 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.913515 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" event={"ID":"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6","Type":"ContainerStarted","Data":"61ada3292805489280ebb2eca4f0dbce446005fc53df6454b73f7d5027740b7d"} Jul 06 22:26:13.915566 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.915542 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" event={"ID":"0a69ad95-6583-444c-add0-abaf3f90c01a","Type":"ContainerStarted","Data":"cd58f9b0231aa9c3911c78ddb38cd6f4dab51bc3a9a129c7bf1c40ee68625b5a"} Jul 06 22:26:13.942030 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:13.942006 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw"] Jul 06 22:26:13.947123 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:26:13.947086 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod903ade62_f582_4c3f_b698_b69b9ada17b8.slice/crio-c4de050b29e701dbcf52e4782833433d1069ac17342862dbf38428f4b4bd0223 WatchSource:0}: Error finding container c4de050b29e701dbcf52e4782833433d1069ac17342862dbf38428f4b4bd0223: Status 404 returned error can't find the container with id c4de050b29e701dbcf52e4782833433d1069ac17342862dbf38428f4b4bd0223 Jul 06 22:26:14.922355 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.922319 2578 generic.go:356] "Generic (PLEG): container finished" podID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerID="db1a4bf1bfef13cdf912739c437470fc0ee8505101dc4c9489816c92ff4e2e8c" exitCode=0 Jul 06 22:26:14.922790 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.922392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" event={"ID":"5142f95c-b34c-47b2-8fbf-f9ad73f39db9","Type":"ContainerDied","Data":"db1a4bf1bfef13cdf912739c437470fc0ee8505101dc4c9489816c92ff4e2e8c"} Jul 06 22:26:14.924001 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.923983 2578 generic.go:356] "Generic (PLEG): container finished" podID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerID="4777956dc36ffe38a5eb65def2d026d02982d3754373ebf17e58b72f51581913" exitCode=0 Jul 06 22:26:14.924093 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.924053 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" event={"ID":"903ade62-f582-4c3f-b698-b69b9ada17b8","Type":"ContainerDied","Data":"4777956dc36ffe38a5eb65def2d026d02982d3754373ebf17e58b72f51581913"} Jul 06 22:26:14.924190 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.924103 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" event={"ID":"903ade62-f582-4c3f-b698-b69b9ada17b8","Type":"ContainerStarted","Data":"c4de050b29e701dbcf52e4782833433d1069ac17342862dbf38428f4b4bd0223"} Jul 06 22:26:14.925973 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.925949 2578 generic.go:356] "Generic (PLEG): container finished" podID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerID="efd8dda6294eee1c6f29e605636486ad47e622c0bd74b31267ba02cc4308fe13" exitCode=0 Jul 06 22:26:14.926090 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.926026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" event={"ID":"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6","Type":"ContainerDied","Data":"efd8dda6294eee1c6f29e605636486ad47e622c0bd74b31267ba02cc4308fe13"} Jul 06 22:26:14.928423 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.928403 2578 generic.go:356] "Generic (PLEG): container finished" podID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerID="cd58f9b0231aa9c3911c78ddb38cd6f4dab51bc3a9a129c7bf1c40ee68625b5a" exitCode=0 Jul 06 22:26:14.928423 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.928421 2578 generic.go:356] "Generic (PLEG): container finished" podID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerID="9b1df8ca56374f1e003d5cb08b4f351e8ec3ee863f4acd7b2d2b16ad9341fe57" exitCode=0 Jul 06 22:26:14.928548 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.928456 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" event={"ID":"0a69ad95-6583-444c-add0-abaf3f90c01a","Type":"ContainerDied","Data":"cd58f9b0231aa9c3911c78ddb38cd6f4dab51bc3a9a129c7bf1c40ee68625b5a"} Jul 06 22:26:14.928548 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:14.928497 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" event={"ID":"0a69ad95-6583-444c-add0-abaf3f90c01a","Type":"ContainerDied","Data":"9b1df8ca56374f1e003d5cb08b4f351e8ec3ee863f4acd7b2d2b16ad9341fe57"} Jul 06 22:26:15.935013 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:15.934981 2578 generic.go:356] "Generic (PLEG): container finished" podID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerID="34288c112dad05fb4c624983e27bbe7d4795e46acb6c1b463f26c86fa7077783" exitCode=0 Jul 06 22:26:15.935435 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:15.935070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" event={"ID":"5142f95c-b34c-47b2-8fbf-f9ad73f39db9","Type":"ContainerDied","Data":"34288c112dad05fb4c624983e27bbe7d4795e46acb6c1b463f26c86fa7077783"} Jul 06 22:26:15.936698 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:15.936679 2578 generic.go:356] "Generic (PLEG): container finished" podID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerID="53e47f52e1d717f57c2a7a52c560ea3266c720d1c29d9db987766824515d7aac" exitCode=0 Jul 06 22:26:15.936755 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:15.936739 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" event={"ID":"903ade62-f582-4c3f-b698-b69b9ada17b8","Type":"ContainerDied","Data":"53e47f52e1d717f57c2a7a52c560ea3266c720d1c29d9db987766824515d7aac"} Jul 06 22:26:15.938813 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:15.938794 2578 generic.go:356] "Generic (PLEG): container finished" podID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerID="a4d459eff8602fc9bf7ad82aa7ccc92fc390123b370c82b70ebd55aafd297881" exitCode=0 Jul 06 22:26:15.938897 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:15.938881 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" event={"ID":"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6","Type":"ContainerDied","Data":"a4d459eff8602fc9bf7ad82aa7ccc92fc390123b370c82b70ebd55aafd297881"} Jul 06 22:26:16.070659 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.070633 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:16.123005 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.122978 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-bundle\") pod \"0a69ad95-6583-444c-add0-abaf3f90c01a\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " Jul 06 22:26:16.123237 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.123027 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-util\") pod \"0a69ad95-6583-444c-add0-abaf3f90c01a\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " Jul 06 22:26:16.123237 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.123069 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkg2v\" (UniqueName: \"kubernetes.io/projected/0a69ad95-6583-444c-add0-abaf3f90c01a-kube-api-access-kkg2v\") pod \"0a69ad95-6583-444c-add0-abaf3f90c01a\" (UID: \"0a69ad95-6583-444c-add0-abaf3f90c01a\") " Jul 06 22:26:16.123522 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.123497 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-bundle" (OuterVolumeSpecName: "bundle") pod "0a69ad95-6583-444c-add0-abaf3f90c01a" (UID: "0a69ad95-6583-444c-add0-abaf3f90c01a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:26:16.125219 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.125191 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a69ad95-6583-444c-add0-abaf3f90c01a-kube-api-access-kkg2v" (OuterVolumeSpecName: "kube-api-access-kkg2v") pod "0a69ad95-6583-444c-add0-abaf3f90c01a" (UID: "0a69ad95-6583-444c-add0-abaf3f90c01a"). InnerVolumeSpecName "kube-api-access-kkg2v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:26:16.128045 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.128024 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-util" (OuterVolumeSpecName: "util") pod "0a69ad95-6583-444c-add0-abaf3f90c01a" (UID: "0a69ad95-6583-444c-add0-abaf3f90c01a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:26:16.223869 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.223829 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-util\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:16.223869 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.223862 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkg2v\" (UniqueName: \"kubernetes.io/projected/0a69ad95-6583-444c-add0-abaf3f90c01a-kube-api-access-kkg2v\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:16.223869 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.223873 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a69ad95-6583-444c-add0-abaf3f90c01a-bundle\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:16.945617 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.945580 2578 generic.go:356] "Generic (PLEG): container finished" podID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerID="21df8676779a5e7f94300c034e337ca752e953419eabec84cca6982d306c223e" exitCode=0 Jul 06 22:26:16.946053 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.945663 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" event={"ID":"903ade62-f582-4c3f-b698-b69b9ada17b8","Type":"ContainerDied","Data":"21df8676779a5e7f94300c034e337ca752e953419eabec84cca6982d306c223e"} Jul 06 22:26:16.947297 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.947271 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" event={"ID":"0a69ad95-6583-444c-add0-abaf3f90c01a","Type":"ContainerDied","Data":"ca7d9a116eed5c494633e30e6a20f58c580445cc1f50d92952725cd6d76ad73a"} Jul 06 22:26:16.947408 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.947314 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca7d9a116eed5c494633e30e6a20f58c580445cc1f50d92952725cd6d76ad73a" Jul 06 22:26:16.947408 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:16.947320 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz" Jul 06 22:26:17.105901 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.105877 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:17.107948 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.107930 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:17.128896 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.128865 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4n58\" (UniqueName: \"kubernetes.io/projected/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-kube-api-access-v4n58\") pod \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " Jul 06 22:26:17.129060 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.128907 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-util\") pod \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " Jul 06 22:26:17.129060 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.128933 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-bundle\") pod \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\" (UID: \"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6\") " Jul 06 22:26:17.129060 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.128964 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-bundle\") pod \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " Jul 06 22:26:17.129060 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.128991 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgv2m\" (UniqueName: \"kubernetes.io/projected/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-kube-api-access-hgv2m\") pod \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " Jul 06 22:26:17.129060 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.129013 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-util\") pod \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\" (UID: \"5142f95c-b34c-47b2-8fbf-f9ad73f39db9\") " Jul 06 22:26:17.129789 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.129750 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-bundle" (OuterVolumeSpecName: "bundle") pod "5142f95c-b34c-47b2-8fbf-f9ad73f39db9" (UID: "5142f95c-b34c-47b2-8fbf-f9ad73f39db9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:26:17.129913 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.129880 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-bundle" (OuterVolumeSpecName: "bundle") pod "5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" (UID: "5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:26:17.131220 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.131188 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-kube-api-access-hgv2m" (OuterVolumeSpecName: "kube-api-access-hgv2m") pod "5142f95c-b34c-47b2-8fbf-f9ad73f39db9" (UID: "5142f95c-b34c-47b2-8fbf-f9ad73f39db9"). InnerVolumeSpecName "kube-api-access-hgv2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:26:17.131336 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.131313 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-kube-api-access-v4n58" (OuterVolumeSpecName: "kube-api-access-v4n58") pod "5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" (UID: "5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6"). InnerVolumeSpecName "kube-api-access-v4n58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:26:17.135521 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.135493 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-util" (OuterVolumeSpecName: "util") pod "5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" (UID: "5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:26:17.136257 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.136219 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-util" (OuterVolumeSpecName: "util") pod "5142f95c-b34c-47b2-8fbf-f9ad73f39db9" (UID: "5142f95c-b34c-47b2-8fbf-f9ad73f39db9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:26:17.230264 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.230219 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4n58\" (UniqueName: \"kubernetes.io/projected/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-kube-api-access-v4n58\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:17.230264 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.230261 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-util\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:17.230264 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.230271 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6-bundle\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:17.230264 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.230279 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-bundle\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:17.230523 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.230287 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgv2m\" (UniqueName: \"kubernetes.io/projected/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-kube-api-access-hgv2m\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:17.230523 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.230296 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5142f95c-b34c-47b2-8fbf-f9ad73f39db9-util\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:17.954378 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.954349 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" Jul 06 22:26:17.954378 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.954365 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs" event={"ID":"5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6","Type":"ContainerDied","Data":"61ada3292805489280ebb2eca4f0dbce446005fc53df6454b73f7d5027740b7d"} Jul 06 22:26:17.954845 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.954417 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61ada3292805489280ebb2eca4f0dbce446005fc53df6454b73f7d5027740b7d" Jul 06 22:26:17.956272 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.956248 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" event={"ID":"5142f95c-b34c-47b2-8fbf-f9ad73f39db9","Type":"ContainerDied","Data":"9776e14698cb27e3dd664df7740d1c25f3d721759d2ed0f4a504364c68227fc8"} Jul 06 22:26:17.956388 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.956288 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9776e14698cb27e3dd664df7740d1c25f3d721759d2ed0f4a504364c68227fc8" Jul 06 22:26:17.956388 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:17.956254 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6" Jul 06 22:26:18.084434 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.084408 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:18.136871 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.136834 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-bundle\") pod \"903ade62-f582-4c3f-b698-b69b9ada17b8\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " Jul 06 22:26:18.136871 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.136872 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-util\") pod \"903ade62-f582-4c3f-b698-b69b9ada17b8\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " Jul 06 22:26:18.137139 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.136907 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtgn5\" (UniqueName: \"kubernetes.io/projected/903ade62-f582-4c3f-b698-b69b9ada17b8-kube-api-access-jtgn5\") pod \"903ade62-f582-4c3f-b698-b69b9ada17b8\" (UID: \"903ade62-f582-4c3f-b698-b69b9ada17b8\") " Jul 06 22:26:18.137442 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.137405 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-bundle" (OuterVolumeSpecName: "bundle") pod "903ade62-f582-4c3f-b698-b69b9ada17b8" (UID: "903ade62-f582-4c3f-b698-b69b9ada17b8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:26:18.138943 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.138921 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/903ade62-f582-4c3f-b698-b69b9ada17b8-kube-api-access-jtgn5" (OuterVolumeSpecName: "kube-api-access-jtgn5") pod "903ade62-f582-4c3f-b698-b69b9ada17b8" (UID: "903ade62-f582-4c3f-b698-b69b9ada17b8"). InnerVolumeSpecName "kube-api-access-jtgn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:26:18.141784 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.141751 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-util" (OuterVolumeSpecName: "util") pod "903ade62-f582-4c3f-b698-b69b9ada17b8" (UID: "903ade62-f582-4c3f-b698-b69b9ada17b8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:26:18.238135 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.238029 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-bundle\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:18.238135 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.238056 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/903ade62-f582-4c3f-b698-b69b9ada17b8-util\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:18.238135 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.238066 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtgn5\" (UniqueName: \"kubernetes.io/projected/903ade62-f582-4c3f-b698-b69b9ada17b8-kube-api-access-jtgn5\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:26:18.962816 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.962787 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" Jul 06 22:26:18.963202 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.962792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw" event={"ID":"903ade62-f582-4c3f-b698-b69b9ada17b8","Type":"ContainerDied","Data":"c4de050b29e701dbcf52e4782833433d1069ac17342862dbf38428f4b4bd0223"} Jul 06 22:26:18.963202 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:18.962895 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4de050b29e701dbcf52e4782833433d1069ac17342862dbf38428f4b4bd0223" Jul 06 22:26:29.304036 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304000 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h546x"] Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304316 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerName="extract" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304327 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerName="extract" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304337 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerName="extract" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304343 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerName="extract" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304350 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerName="util" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304356 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerName="util" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304362 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerName="extract" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304367 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerName="extract" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304374 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerName="extract" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304378 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerName="extract" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304385 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerName="util" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304390 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerName="util" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304396 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerName="util" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304401 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerName="util" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304409 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerName="pull" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304416 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerName="pull" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304422 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerName="pull" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304428 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerName="pull" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304436 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerName="util" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304440 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerName="util" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304446 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerName="pull" Jul 06 22:26:29.304441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304451 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerName="pull" Jul 06 22:26:29.305055 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304457 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerName="pull" Jul 06 22:26:29.305055 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304461 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerName="pull" Jul 06 22:26:29.305055 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304507 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6" containerName="extract" Jul 06 22:26:29.305055 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304514 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a69ad95-6583-444c-add0-abaf3f90c01a" containerName="extract" Jul 06 22:26:29.305055 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304520 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5142f95c-b34c-47b2-8fbf-f9ad73f39db9" containerName="extract" Jul 06 22:26:29.305055 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.304526 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="903ade62-f582-4c3f-b698-b69b9ada17b8" containerName="extract" Jul 06 22:26:29.307645 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.307628 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" Jul 06 22:26:29.310149 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.310103 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xk949\"" Jul 06 22:26:29.318497 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.318470 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h546x"] Jul 06 22:26:29.324363 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.324336 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj585\" (UniqueName: \"kubernetes.io/projected/9dd733e5-551a-4227-9fc5-6ffa7fa589f9-kube-api-access-gj585\") pod \"authorino-operator-75cc8d5cb7-h546x\" (UID: \"9dd733e5-551a-4227-9fc5-6ffa7fa589f9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" Jul 06 22:26:29.424829 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.424795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj585\" (UniqueName: \"kubernetes.io/projected/9dd733e5-551a-4227-9fc5-6ffa7fa589f9-kube-api-access-gj585\") pod \"authorino-operator-75cc8d5cb7-h546x\" (UID: \"9dd733e5-551a-4227-9fc5-6ffa7fa589f9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" Jul 06 22:26:29.435532 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.435507 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj585\" (UniqueName: \"kubernetes.io/projected/9dd733e5-551a-4227-9fc5-6ffa7fa589f9-kube-api-access-gj585\") pod \"authorino-operator-75cc8d5cb7-h546x\" (UID: \"9dd733e5-551a-4227-9fc5-6ffa7fa589f9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" Jul 06 22:26:29.618284 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.618245 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" Jul 06 22:26:29.757516 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:29.757493 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h546x"] Jul 06 22:26:29.759311 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:26:29.759282 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd733e5_551a_4227_9fc5_6ffa7fa589f9.slice/crio-d3c3ed1f12aea3cdbd7289387f230f2d8a8e2910b1a33ab8c40b329b4ed0f26d WatchSource:0}: Error finding container d3c3ed1f12aea3cdbd7289387f230f2d8a8e2910b1a33ab8c40b329b4ed0f26d: Status 404 returned error can't find the container with id d3c3ed1f12aea3cdbd7289387f230f2d8a8e2910b1a33ab8c40b329b4ed0f26d Jul 06 22:26:30.018815 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:30.018727 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" event={"ID":"9dd733e5-551a-4227-9fc5-6ffa7fa589f9","Type":"ContainerStarted","Data":"d3c3ed1f12aea3cdbd7289387f230f2d8a8e2910b1a33ab8c40b329b4ed0f26d"} Jul 06 22:26:32.032696 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.032659 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" event={"ID":"9dd733e5-551a-4227-9fc5-6ffa7fa589f9","Type":"ContainerStarted","Data":"0f922625bb79e8f28f797a6fb0785a690238ac5d67f8bb15aa273c668149f65c"} Jul 06 22:26:32.033076 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.032745 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" Jul 06 22:26:32.050818 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.050715 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" podStartSLOduration=1.013441518 podStartE2EDuration="3.050699761s" totalImagesPullingTime="2.037258243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:26:32.050552493 +0000 UTC m=+382.152280157" watchObservedRunningTime="2026-07-06 22:26:32.050699761 +0000 UTC m=+382.152427423" Jul 06 22:26:32.578136 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.575582 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz"] Jul 06 22:26:32.579857 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.579830 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:26:32.582493 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.582473 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-brcz5\"" Jul 06 22:26:32.591613 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.591592 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz"] Jul 06 22:26:32.652684 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.652648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkbxg\" (UniqueName: \"kubernetes.io/projected/47d5ab36-d966-479b-ba94-2f677ea533f1-kube-api-access-rkbxg\") pod \"limitador-operator-controller-manager-859d7b78bd-kt6zz\" (UID: \"47d5ab36-d966-479b-ba94-2f677ea533f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:26:32.753519 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.753480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkbxg\" (UniqueName: \"kubernetes.io/projected/47d5ab36-d966-479b-ba94-2f677ea533f1-kube-api-access-rkbxg\") pod \"limitador-operator-controller-manager-859d7b78bd-kt6zz\" (UID: \"47d5ab36-d966-479b-ba94-2f677ea533f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:26:32.762224 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.762192 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkbxg\" (UniqueName: \"kubernetes.io/projected/47d5ab36-d966-479b-ba94-2f677ea533f1-kube-api-access-rkbxg\") pod \"limitador-operator-controller-manager-859d7b78bd-kt6zz\" (UID: \"47d5ab36-d966-479b-ba94-2f677ea533f1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:26:32.889685 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:32.889658 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:26:33.036261 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:33.036236 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz"] Jul 06 22:26:33.038831 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:26:33.038806 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d5ab36_d966_479b_ba94_2f677ea533f1.slice/crio-bb1b80d6557ff2953082ffe20ca2e18889167bad495e42c2a1b6a6436bc4f9ae WatchSource:0}: Error finding container bb1b80d6557ff2953082ffe20ca2e18889167bad495e42c2a1b6a6436bc4f9ae: Status 404 returned error can't find the container with id bb1b80d6557ff2953082ffe20ca2e18889167bad495e42c2a1b6a6436bc4f9ae Jul 06 22:26:34.045512 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:34.045468 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" event={"ID":"47d5ab36-d966-479b-ba94-2f677ea533f1","Type":"ContainerStarted","Data":"bb1b80d6557ff2953082ffe20ca2e18889167bad495e42c2a1b6a6436bc4f9ae"} Jul 06 22:26:35.052635 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:35.052599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" event={"ID":"47d5ab36-d966-479b-ba94-2f677ea533f1","Type":"ContainerStarted","Data":"8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1"} Jul 06 22:26:35.053048 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:35.052748 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:26:35.087280 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:35.087230 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" podStartSLOduration=1.237692586 podStartE2EDuration="3.087215132s" totalImagesPullingTime="1.849522546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:26:35.086251092 +0000 UTC m=+385.187978754" watchObservedRunningTime="2026-07-06 22:26:35.087215132 +0000 UTC m=+385.188942794" Jul 06 22:26:39.873108 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:39.873073 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht"] Jul 06 22:26:39.876527 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:39.876511 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" Jul 06 22:26:39.878879 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:39.878855 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 22:26:39.879015 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:39.878919 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5qjc4\"" Jul 06 22:26:39.885872 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:39.885847 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht"] Jul 06 22:26:39.912737 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:39.912710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mszzf\" (UniqueName: \"kubernetes.io/projected/425886c9-a8ad-45a6-be7f-dcfefb8ef3de-kube-api-access-mszzf\") pod \"dns-operator-controller-manager-864dd78c4c-ktzht\" (UID: \"425886c9-a8ad-45a6-be7f-dcfefb8ef3de\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" Jul 06 22:26:40.014026 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:40.013993 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mszzf\" (UniqueName: \"kubernetes.io/projected/425886c9-a8ad-45a6-be7f-dcfefb8ef3de-kube-api-access-mszzf\") pod \"dns-operator-controller-manager-864dd78c4c-ktzht\" (UID: \"425886c9-a8ad-45a6-be7f-dcfefb8ef3de\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" Jul 06 22:26:40.026653 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:40.026626 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mszzf\" (UniqueName: \"kubernetes.io/projected/425886c9-a8ad-45a6-be7f-dcfefb8ef3de-kube-api-access-mszzf\") pod \"dns-operator-controller-manager-864dd78c4c-ktzht\" (UID: \"425886c9-a8ad-45a6-be7f-dcfefb8ef3de\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" Jul 06 22:26:40.187324 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:40.187235 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" Jul 06 22:26:40.321355 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:40.321331 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht"] Jul 06 22:26:40.323415 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:26:40.323382 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod425886c9_a8ad_45a6_be7f_dcfefb8ef3de.slice/crio-32ef969a766ee40d1442a9dc1016b2dd9d79c60a06d78fba19152edac94eb639 WatchSource:0}: Error finding container 32ef969a766ee40d1442a9dc1016b2dd9d79c60a06d78fba19152edac94eb639: Status 404 returned error can't find the container with id 32ef969a766ee40d1442a9dc1016b2dd9d79c60a06d78fba19152edac94eb639 Jul 06 22:26:41.082673 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:41.082641 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" event={"ID":"425886c9-a8ad-45a6-be7f-dcfefb8ef3de","Type":"ContainerStarted","Data":"32ef969a766ee40d1442a9dc1016b2dd9d79c60a06d78fba19152edac94eb639"} Jul 06 22:26:43.040944 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:43.040912 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h546x" Jul 06 22:26:43.095671 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:43.095624 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" event={"ID":"425886c9-a8ad-45a6-be7f-dcfefb8ef3de","Type":"ContainerStarted","Data":"72cb5c11d12fe6649555cd4c0d3508bdb2d38b02742699fee4e00e912cc190da"} Jul 06 22:26:43.095832 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:43.095732 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" Jul 06 22:26:43.124654 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:43.124600 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" podStartSLOduration=2.080646462 podStartE2EDuration="4.124583457s" totalImagesPullingTime="2.043936995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:26:43.122691949 +0000 UTC m=+393.224419611" watchObservedRunningTime="2026-07-06 22:26:43.124583457 +0000 UTC m=+393.226311120" Jul 06 22:26:46.060492 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:46.060462 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:26:47.020923 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.020888 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg"] Jul 06 22:26:47.024958 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.024937 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.027417 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.027393 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fcb48\"" Jul 06 22:26:47.027542 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.027526 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 22:26:47.027956 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.027938 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 22:26:47.033583 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.033562 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg"] Jul 06 22:26:47.072244 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.072207 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.072244 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.072246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b6jf\" (UniqueName: \"kubernetes.io/projected/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-kube-api-access-6b6jf\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.072666 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.072334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.173526 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.173493 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.173719 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.173544 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.173719 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.173572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b6jf\" (UniqueName: \"kubernetes.io/projected/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-kube-api-access-6b6jf\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.173719 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:26:47.173643 2578 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 06 22:26:47.173719 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:26:47.173708 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-plugin-serving-cert podName:9ae7d4f4-56a7-4208-8c4f-1826f4699da5 nodeName:}" failed. No retries permitted until 2026-07-06 22:26:47.67368714 +0000 UTC m=+397.775414790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-nx4bg" (UID: "9ae7d4f4-56a7-4208-8c4f-1826f4699da5") : secret "plugin-serving-cert" not found Jul 06 22:26:47.174407 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.174384 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.208448 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.208412 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b6jf\" (UniqueName: \"kubernetes.io/projected/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-kube-api-access-6b6jf\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.678272 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.678231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.680615 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.680589 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ae7d4f4-56a7-4208-8c4f-1826f4699da5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nx4bg\" (UID: \"9ae7d4f4-56a7-4208-8c4f-1826f4699da5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:47.935680 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:47.935595 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" Jul 06 22:26:48.072822 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:48.072740 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg"] Jul 06 22:26:48.075435 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:26:48.075394 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ae7d4f4_56a7_4208_8c4f_1826f4699da5.slice/crio-79ea679ef7ba6ae513931582882214e7d6942ce975a11897b856e58eb34e460c WatchSource:0}: Error finding container 79ea679ef7ba6ae513931582882214e7d6942ce975a11897b856e58eb34e460c: Status 404 returned error can't find the container with id 79ea679ef7ba6ae513931582882214e7d6942ce975a11897b856e58eb34e460c Jul 06 22:26:48.124042 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:48.124008 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" event={"ID":"9ae7d4f4-56a7-4208-8c4f-1826f4699da5","Type":"ContainerStarted","Data":"79ea679ef7ba6ae513931582882214e7d6942ce975a11897b856e58eb34e460c"} Jul 06 22:26:54.103150 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:54.103095 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ktzht" Jul 06 22:26:56.682271 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.682227 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt"] Jul 06 22:26:56.685819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.685800 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:26:56.688760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.688735 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kpqhd\"" Jul 06 22:26:56.702304 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.702273 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt"] Jul 06 22:26:56.762090 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.762053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtzfx\" (UniqueName: \"kubernetes.io/projected/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-kube-api-access-xtzfx\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" (UID: \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:26:56.762276 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.762227 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" (UID: \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:26:56.863577 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.863545 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" (UID: \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:26:56.863727 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.863594 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtzfx\" (UniqueName: \"kubernetes.io/projected/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-kube-api-access-xtzfx\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" (UID: \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:26:56.863948 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.863927 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" (UID: \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:26:56.880453 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.880427 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtzfx\" (UniqueName: \"kubernetes.io/projected/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-kube-api-access-xtzfx\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" (UID: \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:26:56.996663 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:56.996572 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:26:57.320247 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.320166 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt"] Jul 06 22:26:57.328180 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.328089 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt"] Jul 06 22:26:57.349947 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.349911 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59"] Jul 06 22:26:57.355278 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.355253 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:26:57.369353 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.368728 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz"] Jul 06 22:26:57.369353 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.368948 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" containerName="manager" containerID="cri-o://8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1" gracePeriod=2 Jul 06 22:26:57.373491 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.373458 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59"] Jul 06 22:26:57.376887 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.376862 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz"] Jul 06 22:26:57.393638 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:26:57.393586 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kt6zz\" is forbidden: User \"system:node:ip-10-0-142-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-114.ec2.internal' and this object" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:26:57.409539 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.409502 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr"] Jul 06 22:26:57.409954 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.409933 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" containerName="manager" Jul 06 22:26:57.410045 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.409956 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" containerName="manager" Jul 06 22:26:57.410126 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.410066 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" containerName="manager" Jul 06 22:26:57.413377 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.413354 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" Jul 06 22:26:57.417556 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:26:57.417492 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kt6zz\" is forbidden: User \"system:node:ip-10-0-142-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-114.ec2.internal' and this object" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:26:57.430740 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.430652 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr"] Jul 06 22:26:57.467968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.467916 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lst59\" (UID: \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:26:57.467968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.467967 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7z2g\" (UniqueName: \"kubernetes.io/projected/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-kube-api-access-h7z2g\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lst59\" (UID: \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:26:57.569167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.569121 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lst59\" (UID: \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:26:57.569167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.569168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7z2g\" (UniqueName: \"kubernetes.io/projected/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-kube-api-access-h7z2g\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lst59\" (UID: \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:26:57.569421 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.569258 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swqll\" (UniqueName: \"kubernetes.io/projected/bc1b9d49-257d-448f-8ddb-99b6450362c2-kube-api-access-swqll\") pod \"limitador-operator-controller-manager-859d7b78bd-vg9pr\" (UID: \"bc1b9d49-257d-448f-8ddb-99b6450362c2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" Jul 06 22:26:57.569596 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.569568 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lst59\" (UID: \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:26:57.582248 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.582224 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7z2g\" (UniqueName: \"kubernetes.io/projected/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-kube-api-access-h7z2g\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lst59\" (UID: \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:26:57.670528 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.670487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swqll\" (UniqueName: \"kubernetes.io/projected/bc1b9d49-257d-448f-8ddb-99b6450362c2-kube-api-access-swqll\") pod \"limitador-operator-controller-manager-859d7b78bd-vg9pr\" (UID: \"bc1b9d49-257d-448f-8ddb-99b6450362c2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" Jul 06 22:26:57.670720 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.670562 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:26:57.681553 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.681526 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swqll\" (UniqueName: \"kubernetes.io/projected/bc1b9d49-257d-448f-8ddb-99b6450362c2-kube-api-access-swqll\") pod \"limitador-operator-controller-manager-859d7b78bd-vg9pr\" (UID: \"bc1b9d49-257d-448f-8ddb-99b6450362c2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" Jul 06 22:26:57.749735 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:26:57.749695 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" Jul 06 22:27:11.511751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:11.511720 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:27:11.513960 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:11.513919 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kt6zz\" is forbidden: User \"system:node:ip-10-0-142-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-114.ec2.internal' and this object" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:27:11.527033 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:11.527006 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59"] Jul 06 22:27:11.528316 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:27:11.528290 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c4a362_ce94_4f4c_927b_f0f333c6bcfc.slice/crio-646e8f5df8f6a82d810d9966f19ae3a381e1f577c6962041edbfd258251ef7b6 WatchSource:0}: Error finding container 646e8f5df8f6a82d810d9966f19ae3a381e1f577c6962041edbfd258251ef7b6: Status 404 returned error can't find the container with id 646e8f5df8f6a82d810d9966f19ae3a381e1f577c6962041edbfd258251ef7b6 Jul 06 22:27:11.538651 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:11.538626 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr"] Jul 06 22:27:11.540203 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:27:11.540180 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc1b9d49_257d_448f_8ddb_99b6450362c2.slice/crio-a85e6fa59bfe096e14da37e7bab0161182ae04daaf12b3222f07b8abf04e97fa WatchSource:0}: Error finding container a85e6fa59bfe096e14da37e7bab0161182ae04daaf12b3222f07b8abf04e97fa: Status 404 returned error can't find the container with id a85e6fa59bfe096e14da37e7bab0161182ae04daaf12b3222f07b8abf04e97fa Jul 06 22:27:11.596783 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:11.596761 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkbxg\" (UniqueName: \"kubernetes.io/projected/47d5ab36-d966-479b-ba94-2f677ea533f1-kube-api-access-rkbxg\") pod \"47d5ab36-d966-479b-ba94-2f677ea533f1\" (UID: \"47d5ab36-d966-479b-ba94-2f677ea533f1\") " Jul 06 22:27:11.598605 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:11.598548 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47d5ab36-d966-479b-ba94-2f677ea533f1-kube-api-access-rkbxg" (OuterVolumeSpecName: "kube-api-access-rkbxg") pod "47d5ab36-d966-479b-ba94-2f677ea533f1" (UID: "47d5ab36-d966-479b-ba94-2f677ea533f1"). InnerVolumeSpecName "kube-api-access-rkbxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:27:11.697644 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:11.697607 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkbxg\" (UniqueName: \"kubernetes.io/projected/47d5ab36-d966-479b-ba94-2f677ea533f1-kube-api-access-rkbxg\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:27:12.281275 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.281230 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" event={"ID":"bc1b9d49-257d-448f-8ddb-99b6450362c2","Type":"ContainerStarted","Data":"d432b94a03538fd6b68c19bdc90e78891edcbe531dc6aa1f9dfc0d77d1ebb8f8"} Jul 06 22:27:12.281455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.281303 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" event={"ID":"bc1b9d49-257d-448f-8ddb-99b6450362c2","Type":"ContainerStarted","Data":"a85e6fa59bfe096e14da37e7bab0161182ae04daaf12b3222f07b8abf04e97fa"} Jul 06 22:27:12.281455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.281333 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" Jul 06 22:27:12.282761 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.282701 2578 generic.go:356] "Generic (PLEG): container finished" podID="47d5ab36-d966-479b-ba94-2f677ea533f1" containerID="8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1" exitCode=0 Jul 06 22:27:12.282761 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.282742 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:27:12.282955 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.282809 2578 scope.go:117] "RemoveContainer" containerID="8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1" Jul 06 22:27:12.284717 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.284692 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" event={"ID":"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc","Type":"ContainerStarted","Data":"646e8f5df8f6a82d810d9966f19ae3a381e1f577c6962041edbfd258251ef7b6"} Jul 06 22:27:12.286395 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.286366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" event={"ID":"9ae7d4f4-56a7-4208-8c4f-1826f4699da5","Type":"ContainerStarted","Data":"fbfec9c4f17957ae4a108944721e2d5bbe8ed40e6aa1ae89e3a981df605414d3"} Jul 06 22:27:12.294674 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.294647 2578 scope.go:117] "RemoveContainer" containerID="8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1" Jul 06 22:27:12.294967 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:12.294945 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1\": container with ID starting with 8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1 not found: ID does not exist" containerID="8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1" Jul 06 22:27:12.295026 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.294981 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1"} err="failed to get container status \"8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1\": rpc error: code = NotFound desc = could not find container \"8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1\": container with ID starting with 8fd3e8cc2a8e8e4dd28fc5f84be5d0a9e2638918cca3f02177b972d802c166f1 not found: ID does not exist" Jul 06 22:27:12.310387 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.310343 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" podStartSLOduration=15.310329418 podStartE2EDuration="15.310329418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:26:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:27:12.309498077 +0000 UTC m=+422.411225738" watchObservedRunningTime="2026-07-06 22:27:12.310329418 +0000 UTC m=+422.412057079" Jul 06 22:27:12.312649 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:12.312617 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kt6zz\" is forbidden: User \"system:node:ip-10-0-142-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-114.ec2.internal' and this object" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:27:12.314482 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:12.314453 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kt6zz\" is forbidden: User \"system:node:ip-10-0-142-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-114.ec2.internal' and this object" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kt6zz" Jul 06 22:27:12.332799 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.332731 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nx4bg" podStartSLOduration=2.965846986 podStartE2EDuration="26.332710655s" totalImagesPullingTime="23.366863669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:27:12.331972922 +0000 UTC m=+422.433700580" watchObservedRunningTime="2026-07-06 22:27:12.332710655 +0000 UTC m=+422.434438340" Jul 06 22:27:12.379305 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:12.379268 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47d5ab36-d966-479b-ba94-2f677ea533f1" path="/var/lib/kubelet/pods/47d5ab36-d966-479b-ba94-2f677ea533f1/volumes" Jul 06 22:27:13.904879 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:13.904822 2578 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-s4ttt_kuadrant-system_9479eda8-c6c6-42c2-856c-43f3d3d8ec59_0(c6441891fb2e86435ba4d556c558526ef511413c96dc4aacaba655e19a143727): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-s4ttt to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 06 22:27:13.905230 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:13.904913 2578 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-s4ttt_kuadrant-system_9479eda8-c6c6-42c2-856c-43f3d3d8ec59_0(c6441891fb2e86435ba4d556c558526ef511413c96dc4aacaba655e19a143727): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-s4ttt to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:27:14.297982 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:14.297901 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:27:14.300877 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:14.300844 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" is forbidden: User \"system:node:ip-10-0-142-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-114.ec2.internal' and this object" podUID="9479eda8-c6c6-42c2-856c-43f3d3d8ec59" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:27:14.306225 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:14.306202 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:27:14.309831 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:14.309805 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" is forbidden: User \"system:node:ip-10-0-142-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-114.ec2.internal' and this object" podUID="9479eda8-c6c6-42c2-856c-43f3d3d8ec59" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:27:14.316049 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:14.316029 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtzfx\" (UniqueName: \"kubernetes.io/projected/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-kube-api-access-xtzfx\") pod \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\" (UID: \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\") " Jul 06 22:27:14.316105 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:14.316064 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-extensions-socket-volume\") pod \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\" (UID: \"9479eda8-c6c6-42c2-856c-43f3d3d8ec59\") " Jul 06 22:27:14.316356 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:14.316334 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9479eda8-c6c6-42c2-856c-43f3d3d8ec59" (UID: "9479eda8-c6c6-42c2-856c-43f3d3d8ec59"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:27:14.318209 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:14.318179 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-kube-api-access-xtzfx" (OuterVolumeSpecName: "kube-api-access-xtzfx") pod "9479eda8-c6c6-42c2-856c-43f3d3d8ec59" (UID: "9479eda8-c6c6-42c2-856c-43f3d3d8ec59"). InnerVolumeSpecName "kube-api-access-xtzfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:27:14.386525 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:14.386493 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9479eda8-c6c6-42c2-856c-43f3d3d8ec59" path="/var/lib/kubelet/pods/9479eda8-c6c6-42c2-856c-43f3d3d8ec59/volumes" Jul 06 22:27:14.416936 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:14.416901 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtzfx\" (UniqueName: \"kubernetes.io/projected/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-kube-api-access-xtzfx\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:27:14.416936 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:14.416936 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9479eda8-c6c6-42c2-856c-43f3d3d8ec59-extensions-socket-volume\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:27:15.304759 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:15.304727 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:27:15.307639 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:15.307606 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" is forbidden: User \"system:node:ip-10-0-142-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-114.ec2.internal' and this object" podUID="9479eda8-c6c6-42c2-856c-43f3d3d8ec59" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:27:15.310767 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:27:15.310732 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-s4ttt\" is forbidden: User \"system:node:ip-10-0-142-114.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-114.ec2.internal' and this object" podUID="9479eda8-c6c6-42c2-856c-43f3d3d8ec59" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-s4ttt" Jul 06 22:27:17.315574 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:17.315539 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" event={"ID":"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc","Type":"ContainerStarted","Data":"c1a1ac68106314cb69f980fb7edc6e4b7164f4c31fa1876a4b120cdc18683c14"} Jul 06 22:27:17.316036 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:17.315614 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:27:17.340216 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:17.340164 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" podStartSLOduration=15.615854173 podStartE2EDuration="20.340148387s" totalImagesPullingTime="4.724294214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:27:17.336843729 +0000 UTC m=+427.438571391" watchObservedRunningTime="2026-07-06 22:27:17.340148387 +0000 UTC m=+427.441876049" Jul 06 22:27:23.295625 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:23.295591 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vg9pr" Jul 06 22:27:28.322356 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:28.322326 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:27:40.671972 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:40.671891 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59"] Jul 06 22:27:40.672359 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:40.672169 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" podUID="d0c4a362-ce94-4f4c-927b-f0f333c6bcfc" containerName="manager" containerID="cri-o://c1a1ac68106314cb69f980fb7edc6e4b7164f4c31fa1876a4b120cdc18683c14" gracePeriod=10 Jul 06 22:27:41.440098 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:41.440068 2578 generic.go:356] "Generic (PLEG): container finished" podID="d0c4a362-ce94-4f4c-927b-f0f333c6bcfc" containerID="c1a1ac68106314cb69f980fb7edc6e4b7164f4c31fa1876a4b120cdc18683c14" exitCode=0 Jul 06 22:27:41.440262 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:41.440155 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" event={"ID":"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc","Type":"ContainerDied","Data":"c1a1ac68106314cb69f980fb7edc6e4b7164f4c31fa1876a4b120cdc18683c14"} Jul 06 22:27:41.537376 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:41.537340 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:27:41.627150 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:41.627036 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-extensions-socket-volume\") pod \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\" (UID: \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\") " Jul 06 22:27:41.627150 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:41.627134 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7z2g\" (UniqueName: \"kubernetes.io/projected/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-kube-api-access-h7z2g\") pod \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\" (UID: \"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc\") " Jul 06 22:27:41.627613 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:41.627590 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d0c4a362-ce94-4f4c-927b-f0f333c6bcfc" (UID: "d0c4a362-ce94-4f4c-927b-f0f333c6bcfc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:27:41.629221 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:41.629195 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-kube-api-access-h7z2g" (OuterVolumeSpecName: "kube-api-access-h7z2g") pod "d0c4a362-ce94-4f4c-927b-f0f333c6bcfc" (UID: "d0c4a362-ce94-4f4c-927b-f0f333c6bcfc"). InnerVolumeSpecName "kube-api-access-h7z2g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:27:41.728017 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:41.727976 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7z2g\" (UniqueName: \"kubernetes.io/projected/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-kube-api-access-h7z2g\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:27:41.728017 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:41.728012 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc-extensions-socket-volume\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:27:42.446763 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:42.446731 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" Jul 06 22:27:42.446763 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:42.446755 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59" event={"ID":"d0c4a362-ce94-4f4c-927b-f0f333c6bcfc","Type":"ContainerDied","Data":"646e8f5df8f6a82d810d9966f19ae3a381e1f577c6962041edbfd258251ef7b6"} Jul 06 22:27:42.446998 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:42.446818 2578 scope.go:117] "RemoveContainer" containerID="c1a1ac68106314cb69f980fb7edc6e4b7164f4c31fa1876a4b120cdc18683c14" Jul 06 22:27:42.477475 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:42.476099 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59"] Jul 06 22:27:42.481730 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:42.481703 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lst59"] Jul 06 22:27:44.379022 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.378980 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0c4a362-ce94-4f4c-927b-f0f333c6bcfc" path="/var/lib/kubelet/pods/d0c4a362-ce94-4f4c-927b-f0f333c6bcfc/volumes" Jul 06 22:27:44.781782 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.781707 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx"] Jul 06 22:27:44.782034 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.782023 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0c4a362-ce94-4f4c-927b-f0f333c6bcfc" containerName="manager" Jul 06 22:27:44.782075 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.782035 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0c4a362-ce94-4f4c-927b-f0f333c6bcfc" containerName="manager" Jul 06 22:27:44.782121 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.782089 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0c4a362-ce94-4f4c-927b-f0f333c6bcfc" containerName="manager" Jul 06 22:27:44.809425 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.809397 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx"] Jul 06 22:27:44.809564 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.809527 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.812621 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.812598 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-f6tt5\"" Jul 06 22:27:44.813151 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.813129 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 22:27:44.856340 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.856306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.856340 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.856338 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.856536 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.856362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97ghm\" (UniqueName: \"kubernetes.io/projected/a677d65b-4c54-498f-9514-d94a9d760563-kube-api-access-97ghm\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.856536 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.856467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.856536 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.856496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.856649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.856557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a677d65b-4c54-498f-9514-d94a9d760563-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.856649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.856582 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a677d65b-4c54-498f-9514-d94a9d760563-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.856751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.856640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.856751 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.856679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a677d65b-4c54-498f-9514-d94a9d760563-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.957730 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.957691 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.957932 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.957745 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a677d65b-4c54-498f-9514-d94a9d760563-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.957932 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.957779 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a677d65b-4c54-498f-9514-d94a9d760563-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.957932 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.957810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.957932 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.957833 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a677d65b-4c54-498f-9514-d94a9d760563-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.957932 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.957873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.957932 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.957896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.957932 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.957929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97ghm\" (UniqueName: \"kubernetes.io/projected/a677d65b-4c54-498f-9514-d94a9d760563-kube-api-access-97ghm\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.958326 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.957986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.958326 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.958198 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.958326 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.958264 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.958504 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.958480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.958560 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.958519 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.958804 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.958780 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a677d65b-4c54-498f-9514-d94a9d760563-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.960235 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.960215 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a677d65b-4c54-498f-9514-d94a9d760563-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.960639 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.960622 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a677d65b-4c54-498f-9514-d94a9d760563-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.972879 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.972855 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97ghm\" (UniqueName: \"kubernetes.io/projected/a677d65b-4c54-498f-9514-d94a9d760563-kube-api-access-97ghm\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:44.973259 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:44.973234 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a677d65b-4c54-498f-9514-d94a9d760563-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-ddkkx\" (UID: \"a677d65b-4c54-498f-9514-d94a9d760563\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:45.120299 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:45.120265 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:45.262289 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:45.262266 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx"] Jul 06 22:27:45.264552 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:27:45.264525 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda677d65b_4c54_498f_9514_d94a9d760563.slice/crio-6bb38d7d80632c73c71bc84507bd5ed429ba521ea6ea8fdcc8044de49f0a3e65 WatchSource:0}: Error finding container 6bb38d7d80632c73c71bc84507bd5ed429ba521ea6ea8fdcc8044de49f0a3e65: Status 404 returned error can't find the container with id 6bb38d7d80632c73c71bc84507bd5ed429ba521ea6ea8fdcc8044de49f0a3e65 Jul 06 22:27:45.466285 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:45.466194 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" event={"ID":"a677d65b-4c54-498f-9514-d94a9d760563","Type":"ContainerStarted","Data":"6bb38d7d80632c73c71bc84507bd5ed429ba521ea6ea8fdcc8044de49f0a3e65"} Jul 06 22:27:47.714623 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:47.714585 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:27:47.714904 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:47.714661 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:27:47.714904 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:47.714688 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:27:48.484571 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:48.484526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" event={"ID":"a677d65b-4c54-498f-9514-d94a9d760563","Type":"ContainerStarted","Data":"cc0f6c3e2ed7419d09832ccaa8c4fd0f3f84edd1cce37d3de61b71ce116c7a47"} Jul 06 22:27:48.507554 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:48.507493 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" podStartSLOduration=2.059734551 podStartE2EDuration="4.507472949s" totalImagesPullingTime="2.447738398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:27:48.507041218 +0000 UTC m=+458.608768881" watchObservedRunningTime="2026-07-06 22:27:48.507472949 +0000 UTC m=+458.609200622" Jul 06 22:27:49.120459 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.120420 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:49.125753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.125725 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:49.490317 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.490243 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:49.491460 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.491441 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:27:49.647142 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.647074 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kjrp9"] Jul 06 22:27:49.650052 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.650027 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:27:49.653781 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.653757 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 22:27:49.660826 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.660802 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kjrp9"] Jul 06 22:27:49.700258 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.700221 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-config-file\") pod \"limitador-limitador-64c94fd64f-kjrp9\" (UID: \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:27:49.700449 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.700282 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd9dn\" (UniqueName: \"kubernetes.io/projected/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-kube-api-access-sd9dn\") pod \"limitador-limitador-64c94fd64f-kjrp9\" (UID: \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:27:49.749413 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.749326 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kjrp9"] Jul 06 22:27:49.801051 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.801017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd9dn\" (UniqueName: \"kubernetes.io/projected/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-kube-api-access-sd9dn\") pod \"limitador-limitador-64c94fd64f-kjrp9\" (UID: \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:27:49.801263 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.801101 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-config-file\") pod \"limitador-limitador-64c94fd64f-kjrp9\" (UID: \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:27:49.801719 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.801700 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-config-file\") pod \"limitador-limitador-64c94fd64f-kjrp9\" (UID: \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:27:49.810584 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.810553 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd9dn\" (UniqueName: \"kubernetes.io/projected/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-kube-api-access-sd9dn\") pod \"limitador-limitador-64c94fd64f-kjrp9\" (UID: \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:27:49.960128 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:49.960083 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:27:50.117018 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:50.116988 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kjrp9"] Jul 06 22:27:50.117911 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:27:50.117883 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1bae34e_eb09_4261_9b16_a6b129bf2a7e.slice/crio-6f5cdee998829bfaacb6b00b6980866c59e88ccbf1e6bce9a61adf728c4bf8a6 WatchSource:0}: Error finding container 6f5cdee998829bfaacb6b00b6980866c59e88ccbf1e6bce9a61adf728c4bf8a6: Status 404 returned error can't find the container with id 6f5cdee998829bfaacb6b00b6980866c59e88ccbf1e6bce9a61adf728c4bf8a6 Jul 06 22:27:50.496304 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:50.496262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" event={"ID":"c1bae34e-eb09-4261-9b16-a6b129bf2a7e","Type":"ContainerStarted","Data":"6f5cdee998829bfaacb6b00b6980866c59e88ccbf1e6bce9a61adf728c4bf8a6"} Jul 06 22:27:53.514127 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:53.514078 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" event={"ID":"c1bae34e-eb09-4261-9b16-a6b129bf2a7e","Type":"ContainerStarted","Data":"269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358"} Jul 06 22:27:53.514521 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:53.514268 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:27:53.534437 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:27:53.534380 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" podStartSLOduration=1.757881553 podStartE2EDuration="4.534363645s" totalImagesPullingTime="2.776482092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:27:53.532866817 +0000 UTC m=+463.634594479" watchObservedRunningTime="2026-07-06 22:27:53.534363645 +0000 UTC m=+463.636091307" Jul 06 22:28:04.520549 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:04.520517 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:28:05.034885 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.034847 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kjrp9"] Jul 06 22:28:05.035081 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.035058 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" podUID="c1bae34e-eb09-4261-9b16-a6b129bf2a7e" containerName="limitador" containerID="cri-o://269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358" gracePeriod=30 Jul 06 22:28:05.576711 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.576681 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:28:05.577824 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.577800 2578 generic.go:356] "Generic (PLEG): container finished" podID="c1bae34e-eb09-4261-9b16-a6b129bf2a7e" containerID="269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358" exitCode=0 Jul 06 22:28:05.577940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.577825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" event={"ID":"c1bae34e-eb09-4261-9b16-a6b129bf2a7e","Type":"ContainerDied","Data":"269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358"} Jul 06 22:28:05.577940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.577874 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" event={"ID":"c1bae34e-eb09-4261-9b16-a6b129bf2a7e","Type":"ContainerDied","Data":"6f5cdee998829bfaacb6b00b6980866c59e88ccbf1e6bce9a61adf728c4bf8a6"} Jul 06 22:28:05.577940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.577898 2578 scope.go:117] "RemoveContainer" containerID="269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358" Jul 06 22:28:05.586067 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.586050 2578 scope.go:117] "RemoveContainer" containerID="269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358" Jul 06 22:28:05.586346 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:28:05.586328 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358\": container with ID starting with 269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358 not found: ID does not exist" containerID="269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358" Jul 06 22:28:05.586386 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.586358 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358"} err="failed to get container status \"269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358\": rpc error: code = NotFound desc = could not find container \"269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358\": container with ID starting with 269cfb7548f1d0e52867fad455360a0fc59fde61e3cfe77fe759d07ce0bff358 not found: ID does not exist" Jul 06 22:28:05.741633 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.741550 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-config-file\") pod \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\" (UID: \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\") " Jul 06 22:28:05.741633 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.741583 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd9dn\" (UniqueName: \"kubernetes.io/projected/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-kube-api-access-sd9dn\") pod \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\" (UID: \"c1bae34e-eb09-4261-9b16-a6b129bf2a7e\") " Jul 06 22:28:05.741897 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.741873 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-config-file" (OuterVolumeSpecName: "config-file") pod "c1bae34e-eb09-4261-9b16-a6b129bf2a7e" (UID: "c1bae34e-eb09-4261-9b16-a6b129bf2a7e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:28:05.743649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.743615 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-kube-api-access-sd9dn" (OuterVolumeSpecName: "kube-api-access-sd9dn") pod "c1bae34e-eb09-4261-9b16-a6b129bf2a7e" (UID: "c1bae34e-eb09-4261-9b16-a6b129bf2a7e"). InnerVolumeSpecName "kube-api-access-sd9dn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:28:05.842812 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.842772 2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-config-file\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:28:05.842812 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:05.842807 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sd9dn\" (UniqueName: \"kubernetes.io/projected/c1bae34e-eb09-4261-9b16-a6b129bf2a7e-kube-api-access-sd9dn\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:28:06.583819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:06.583787 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kjrp9" Jul 06 22:28:06.605491 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:06.605456 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kjrp9"] Jul 06 22:28:06.610442 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:06.610414 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kjrp9"] Jul 06 22:28:08.378557 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:08.378526 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1bae34e-eb09-4261-9b16-a6b129bf2a7e" path="/var/lib/kubelet/pods/c1bae34e-eb09-4261-9b16-a6b129bf2a7e/volumes" Jul 06 22:28:11.153511 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.153475 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-hjp5x"] Jul 06 22:28:11.153939 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.153884 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1bae34e-eb09-4261-9b16-a6b129bf2a7e" containerName="limitador" Jul 06 22:28:11.153939 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.153899 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1bae34e-eb09-4261-9b16-a6b129bf2a7e" containerName="limitador" Jul 06 22:28:11.154054 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.153983 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1bae34e-eb09-4261-9b16-a6b129bf2a7e" containerName="limitador" Jul 06 22:28:11.158051 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.158033 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:11.160628 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.160601 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 22:28:11.160628 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.160601 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s66pf\"" Jul 06 22:28:11.166495 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.166467 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-hjp5x"] Jul 06 22:28:11.286438 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.286401 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2kld\" (UniqueName: \"kubernetes.io/projected/2e35f3a7-c841-47db-86a3-0d18a6924512-kube-api-access-g2kld\") pod \"postgres-69fb4594cf-hjp5x\" (UID: \"2e35f3a7-c841-47db-86a3-0d18a6924512\") " pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:11.286438 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.286447 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2e35f3a7-c841-47db-86a3-0d18a6924512-data\") pod \"postgres-69fb4594cf-hjp5x\" (UID: \"2e35f3a7-c841-47db-86a3-0d18a6924512\") " pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:11.387097 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.387069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2kld\" (UniqueName: \"kubernetes.io/projected/2e35f3a7-c841-47db-86a3-0d18a6924512-kube-api-access-g2kld\") pod \"postgres-69fb4594cf-hjp5x\" (UID: \"2e35f3a7-c841-47db-86a3-0d18a6924512\") " pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:11.387097 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.387103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2e35f3a7-c841-47db-86a3-0d18a6924512-data\") pod \"postgres-69fb4594cf-hjp5x\" (UID: \"2e35f3a7-c841-47db-86a3-0d18a6924512\") " pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:11.387455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.387439 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2e35f3a7-c841-47db-86a3-0d18a6924512-data\") pod \"postgres-69fb4594cf-hjp5x\" (UID: \"2e35f3a7-c841-47db-86a3-0d18a6924512\") " pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:11.397036 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.397009 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2kld\" (UniqueName: \"kubernetes.io/projected/2e35f3a7-c841-47db-86a3-0d18a6924512-kube-api-access-g2kld\") pod \"postgres-69fb4594cf-hjp5x\" (UID: \"2e35f3a7-c841-47db-86a3-0d18a6924512\") " pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:11.470166 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.470061 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:11.605489 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:11.605456 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-hjp5x"] Jul 06 22:28:11.606980 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:28:11.606953 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e35f3a7_c841_47db_86a3_0d18a6924512.slice/crio-4c7986dde1c2a3992fd630a535304bd8d5f1134e98012cd27fb34eb2ac1d9105 WatchSource:0}: Error finding container 4c7986dde1c2a3992fd630a535304bd8d5f1134e98012cd27fb34eb2ac1d9105: Status 404 returned error can't find the container with id 4c7986dde1c2a3992fd630a535304bd8d5f1134e98012cd27fb34eb2ac1d9105 Jul 06 22:28:12.619248 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:12.619209 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-hjp5x" event={"ID":"2e35f3a7-c841-47db-86a3-0d18a6924512","Type":"ContainerStarted","Data":"4c7986dde1c2a3992fd630a535304bd8d5f1134e98012cd27fb34eb2ac1d9105"} Jul 06 22:28:16.644061 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:16.644021 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-hjp5x" event={"ID":"2e35f3a7-c841-47db-86a3-0d18a6924512","Type":"ContainerStarted","Data":"03463276ebc64163b82ceccaea3f242d2b12fe53a8029c3bcb28fec94d5153ad"} Jul 06 22:28:16.644473 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:16.644101 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:16.666366 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:16.666267 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-hjp5x" podStartSLOduration=0.857440188 podStartE2EDuration="5.666251494s" totalImagesPullingTime="4.808811306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:28:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:28:16.664735197 +0000 UTC m=+486.766462860" watchObservedRunningTime="2026-07-06 22:28:16.666251494 +0000 UTC m=+486.767979170" Jul 06 22:28:22.678955 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:22.678924 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-hjp5x" Jul 06 22:28:23.586601 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.586568 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-648cb8c54c-hb649"] Jul 06 22:28:23.595020 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.594985 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-648cb8c54c-hb649" Jul 06 22:28:23.598209 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.598177 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-f9fwm\"" Jul 06 22:28:23.601682 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.601656 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-648cb8c54c-hb649"] Jul 06 22:28:23.678398 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.678366 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-648cb8c54c-hb649"] Jul 06 22:28:23.678635 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:28:23.678613 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-dsg75], unattached volumes=[], failed to process volumes=[kube-api-access-dsg75]: context canceled" pod="kuadrant-system/authorino-648cb8c54c-hb649" podUID="c064653d-3e39-44dc-865a-448596321284" Jul 06 22:28:23.684682 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.684645 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsg75\" (UniqueName: \"kubernetes.io/projected/c064653d-3e39-44dc-865a-448596321284-kube-api-access-dsg75\") pod \"authorino-648cb8c54c-hb649\" (UID: \"c064653d-3e39-44dc-865a-448596321284\") " pod="kuadrant-system/authorino-648cb8c54c-hb649" Jul 06 22:28:23.712145 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.712101 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-578bd64fc5-8nsrq"] Jul 06 22:28:23.715966 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.715944 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:28:23.718728 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.718708 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 22:28:23.726515 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.726495 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-578bd64fc5-8nsrq"] Jul 06 22:28:23.786191 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.786147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e808e501-1581-40dd-9a64-0b228ac751f1-tls-cert\") pod \"authorino-578bd64fc5-8nsrq\" (UID: \"e808e501-1581-40dd-9a64-0b228ac751f1\") " pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:28:23.786373 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.786245 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp6p8\" (UniqueName: \"kubernetes.io/projected/e808e501-1581-40dd-9a64-0b228ac751f1-kube-api-access-pp6p8\") pod \"authorino-578bd64fc5-8nsrq\" (UID: \"e808e501-1581-40dd-9a64-0b228ac751f1\") " pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:28:23.786373 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.786279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsg75\" (UniqueName: \"kubernetes.io/projected/c064653d-3e39-44dc-865a-448596321284-kube-api-access-dsg75\") pod \"authorino-648cb8c54c-hb649\" (UID: \"c064653d-3e39-44dc-865a-448596321284\") " pod="kuadrant-system/authorino-648cb8c54c-hb649" Jul 06 22:28:23.796985 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.796949 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsg75\" (UniqueName: \"kubernetes.io/projected/c064653d-3e39-44dc-865a-448596321284-kube-api-access-dsg75\") pod \"authorino-648cb8c54c-hb649\" (UID: \"c064653d-3e39-44dc-865a-448596321284\") " pod="kuadrant-system/authorino-648cb8c54c-hb649" Jul 06 22:28:23.887155 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.887020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e808e501-1581-40dd-9a64-0b228ac751f1-tls-cert\") pod \"authorino-578bd64fc5-8nsrq\" (UID: \"e808e501-1581-40dd-9a64-0b228ac751f1\") " pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:28:23.887155 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.887092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp6p8\" (UniqueName: \"kubernetes.io/projected/e808e501-1581-40dd-9a64-0b228ac751f1-kube-api-access-pp6p8\") pod \"authorino-578bd64fc5-8nsrq\" (UID: \"e808e501-1581-40dd-9a64-0b228ac751f1\") " pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:28:23.889525 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.889496 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e808e501-1581-40dd-9a64-0b228ac751f1-tls-cert\") pod \"authorino-578bd64fc5-8nsrq\" (UID: \"e808e501-1581-40dd-9a64-0b228ac751f1\") " pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:28:23.897705 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:23.897671 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp6p8\" (UniqueName: \"kubernetes.io/projected/e808e501-1581-40dd-9a64-0b228ac751f1-kube-api-access-pp6p8\") pod \"authorino-578bd64fc5-8nsrq\" (UID: \"e808e501-1581-40dd-9a64-0b228ac751f1\") " pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:28:24.025044 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:24.025000 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:28:24.165147 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:24.165105 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-578bd64fc5-8nsrq"] Jul 06 22:28:24.167412 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:28:24.167386 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode808e501_1581_40dd_9a64_0b228ac751f1.slice/crio-9f0c6fdd33c49a05656d1c87899683dae4f042904b02c885187431ae3fd450c1 WatchSource:0}: Error finding container 9f0c6fdd33c49a05656d1c87899683dae4f042904b02c885187431ae3fd450c1: Status 404 returned error can't find the container with id 9f0c6fdd33c49a05656d1c87899683dae4f042904b02c885187431ae3fd450c1 Jul 06 22:28:24.688978 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:24.688944 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-648cb8c54c-hb649" Jul 06 22:28:24.689482 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:24.689022 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" event={"ID":"e808e501-1581-40dd-9a64-0b228ac751f1","Type":"ContainerStarted","Data":"9f0c6fdd33c49a05656d1c87899683dae4f042904b02c885187431ae3fd450c1"} Jul 06 22:28:24.699305 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:24.699277 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-648cb8c54c-hb649" Jul 06 22:28:24.794594 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:24.794558 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsg75\" (UniqueName: \"kubernetes.io/projected/c064653d-3e39-44dc-865a-448596321284-kube-api-access-dsg75\") pod \"c064653d-3e39-44dc-865a-448596321284\" (UID: \"c064653d-3e39-44dc-865a-448596321284\") " Jul 06 22:28:24.797670 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:24.797623 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c064653d-3e39-44dc-865a-448596321284-kube-api-access-dsg75" (OuterVolumeSpecName: "kube-api-access-dsg75") pod "c064653d-3e39-44dc-865a-448596321284" (UID: "c064653d-3e39-44dc-865a-448596321284"). InnerVolumeSpecName "kube-api-access-dsg75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:28:24.895136 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:24.895075 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsg75\" (UniqueName: \"kubernetes.io/projected/c064653d-3e39-44dc-865a-448596321284-kube-api-access-dsg75\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:28:25.701059 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:25.701021 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-648cb8c54c-hb649" Jul 06 22:28:25.738660 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:25.738626 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-648cb8c54c-hb649"] Jul 06 22:28:25.742726 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:25.742686 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-648cb8c54c-hb649"] Jul 06 22:28:26.380465 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:26.380429 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c064653d-3e39-44dc-865a-448596321284" path="/var/lib/kubelet/pods/c064653d-3e39-44dc-865a-448596321284/volumes" Jul 06 22:28:27.714030 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:27.713946 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" event={"ID":"e808e501-1581-40dd-9a64-0b228ac751f1","Type":"ContainerStarted","Data":"04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e"} Jul 06 22:28:27.736712 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:28:27.736660 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" podStartSLOduration=1.543245827 podStartE2EDuration="4.736646227s" totalImagesPullingTime="3.1934004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:28:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:28:27.735067334 +0000 UTC m=+497.836794996" watchObservedRunningTime="2026-07-06 22:28:27.736646227 +0000 UTC m=+497.838373889" Jul 06 22:29:10.857052 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:10.857016 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-q2df4"] Jul 06 22:29:10.860581 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:10.860561 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:10.862899 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:10.862878 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8b5qj\"" Jul 06 22:29:10.863008 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:10.862929 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 22:29:10.871127 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:10.871090 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-q2df4"] Jul 06 22:29:10.979785 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:10.979754 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6q2l\" (UniqueName: \"kubernetes.io/projected/8c2a0878-3414-4c47-a3d4-502f8ded2a4b-kube-api-access-j6q2l\") pod \"payload-pre-processing-7964457696-q2df4\" (UID: \"8c2a0878-3414-4c47-a3d4-502f8ded2a4b\") " pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:10.979944 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:10.979798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c2a0878-3414-4c47-a3d4-502f8ded2a4b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-q2df4\" (UID: \"8c2a0878-3414-4c47-a3d4-502f8ded2a4b\") " pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:11.080267 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:11.080227 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6q2l\" (UniqueName: \"kubernetes.io/projected/8c2a0878-3414-4c47-a3d4-502f8ded2a4b-kube-api-access-j6q2l\") pod \"payload-pre-processing-7964457696-q2df4\" (UID: \"8c2a0878-3414-4c47-a3d4-502f8ded2a4b\") " pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:11.080443 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:11.080279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c2a0878-3414-4c47-a3d4-502f8ded2a4b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-q2df4\" (UID: \"8c2a0878-3414-4c47-a3d4-502f8ded2a4b\") " pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:11.081025 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:11.081003 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c2a0878-3414-4c47-a3d4-502f8ded2a4b-plugins-config-volume\") pod \"payload-pre-processing-7964457696-q2df4\" (UID: \"8c2a0878-3414-4c47-a3d4-502f8ded2a4b\") " pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:11.089941 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:11.089908 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6q2l\" (UniqueName: \"kubernetes.io/projected/8c2a0878-3414-4c47-a3d4-502f8ded2a4b-kube-api-access-j6q2l\") pod \"payload-pre-processing-7964457696-q2df4\" (UID: \"8c2a0878-3414-4c47-a3d4-502f8ded2a4b\") " pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:11.171996 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:11.171898 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:11.311550 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:11.311509 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-q2df4"] Jul 06 22:29:11.314532 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:29:11.314494 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2a0878_3414_4c47_a3d4_502f8ded2a4b.slice/crio-d5f04d25cb322363cab317662de9cd61cfd0cc7d2b305a78e973d313aaadd322 WatchSource:0}: Error finding container d5f04d25cb322363cab317662de9cd61cfd0cc7d2b305a78e973d313aaadd322: Status 404 returned error can't find the container with id d5f04d25cb322363cab317662de9cd61cfd0cc7d2b305a78e973d313aaadd322 Jul 06 22:29:11.941963 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:11.941921 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" event={"ID":"8c2a0878-3414-4c47-a3d4-502f8ded2a4b","Type":"ContainerStarted","Data":"d5f04d25cb322363cab317662de9cd61cfd0cc7d2b305a78e973d313aaadd322"} Jul 06 22:29:12.948372 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:12.948336 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" event={"ID":"8c2a0878-3414-4c47-a3d4-502f8ded2a4b","Type":"ContainerStarted","Data":"4e0fc5a345833dc31326b83bbce1fd9348dc3c0aed15e12142ab7a8881cf95c4"} Jul 06 22:29:12.948749 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:12.948431 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:12.974149 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:12.972260 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" podStartSLOduration=1.429915744 podStartE2EDuration="2.972236392s" totalImagesPullingTime="1.542320648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:29:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:29:12.970515042 +0000 UTC m=+543.072242705" watchObservedRunningTime="2026-07-06 22:29:12.972236392 +0000 UTC m=+543.073964055" Jul 06 22:29:18.984443 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:18.984406 2578 generic.go:356] "Generic (PLEG): container finished" podID="a677d65b-4c54-498f-9514-d94a9d760563" containerID="cc0f6c3e2ed7419d09832ccaa8c4fd0f3f84edd1cce37d3de61b71ce116c7a47" exitCode=0 Jul 06 22:29:18.984857 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:18.984479 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" event={"ID":"a677d65b-4c54-498f-9514-d94a9d760563","Type":"ContainerDied","Data":"cc0f6c3e2ed7419d09832ccaa8c4fd0f3f84edd1cce37d3de61b71ce116c7a47"} Jul 06 22:29:18.984898 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:18.984860 2578 scope.go:117] "RemoveContainer" containerID="cc0f6c3e2ed7419d09832ccaa8c4fd0f3f84edd1cce37d3de61b71ce116c7a47" Jul 06 22:29:18.985842 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:18.985809 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:29:18.985937 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:18.985887 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:29:18.985937 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:18.985923 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:29:19.490508 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:19.490472 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:29:19.993023 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:19.992990 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" event={"ID":"a677d65b-4c54-498f-9514-d94a9d760563","Type":"ContainerStarted","Data":"fa514c99fb7b694999074e4a0df7500a7ff73eee782c7ff92aeaa8632917da5c"} Jul 06 22:29:20.120776 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:20.120737 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:29:20.121896 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:20.121867 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-ddkkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 06 22:29:20.122002 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:20.121920 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" podUID="a677d65b-4c54-498f-9514-d94a9d760563" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 06 22:29:21.120666 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:21.120630 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-ddkkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 06 22:29:21.121066 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:21.120696 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" podUID="a677d65b-4c54-498f-9514-d94a9d760563" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 06 22:29:22.121053 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:22.121005 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-ddkkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 06 22:29:22.121511 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:22.121077 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" podUID="a677d65b-4c54-498f-9514-d94a9d760563" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 06 22:29:23.121280 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:23.121240 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-ddkkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 06 22:29:23.121686 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:23.121296 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" podUID="a677d65b-4c54-498f-9514-d94a9d760563" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 06 22:29:23.956649 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:23.956619 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-q2df4" Jul 06 22:29:24.125345 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:24.125312 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:29:24.125732 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:24.125594 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:29:24.126271 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:24.126255 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-ddkkx" Jul 06 22:29:31.426131 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.426083 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd"] Jul 06 22:29:31.430819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.430799 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.433945 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.433893 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 22:29:31.434072 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.433928 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-j9vrr\"" Jul 06 22:29:31.434072 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.434002 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 22:29:31.434215 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.434084 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 22:29:31.439358 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.439335 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd"] Jul 06 22:29:31.458845 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.458812 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv7fg\" (UniqueName: \"kubernetes.io/projected/62400b3f-8bd9-497c-b400-a794e6c98b08-kube-api-access-nv7fg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.459034 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.458850 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.459034 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.458933 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.459034 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.459001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.459185 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.459058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.459185 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.459088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62400b3f-8bd9-497c-b400-a794e6c98b08-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.559484 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.559441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.559678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.559504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.559678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.559524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.559678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.559539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62400b3f-8bd9-497c-b400-a794e6c98b08-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.559678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.559568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv7fg\" (UniqueName: \"kubernetes.io/projected/62400b3f-8bd9-497c-b400-a794e6c98b08-kube-api-access-nv7fg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.559678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.559594 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.559942 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.559900 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.559998 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.559943 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.560051 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.560014 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.561887 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.561864 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62400b3f-8bd9-497c-b400-a794e6c98b08-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.562197 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.562174 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62400b3f-8bd9-497c-b400-a794e6c98b08-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.568675 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.568650 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv7fg\" (UniqueName: \"kubernetes.io/projected/62400b3f-8bd9-497c-b400-a794e6c98b08-kube-api-access-nv7fg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-nzrfd\" (UID: \"62400b3f-8bd9-497c-b400-a794e6c98b08\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.741930 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.741848 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:31.879974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:31.879950 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd"] Jul 06 22:29:31.881900 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:29:31.881874 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62400b3f_8bd9_497c_b400_a794e6c98b08.slice/crio-a228de446295c80d1b039c772249bce3e76ac482f9d168c644766c037788a822 WatchSource:0}: Error finding container a228de446295c80d1b039c772249bce3e76ac482f9d168c644766c037788a822: Status 404 returned error can't find the container with id a228de446295c80d1b039c772249bce3e76ac482f9d168c644766c037788a822 Jul 06 22:29:32.056911 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:32.056827 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" event={"ID":"62400b3f-8bd9-497c-b400-a794e6c98b08","Type":"ContainerStarted","Data":"a228de446295c80d1b039c772249bce3e76ac482f9d168c644766c037788a822"} Jul 06 22:29:33.071200 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:33.071153 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" event={"ID":"62400b3f-8bd9-497c-b400-a794e6c98b08","Type":"ContainerStarted","Data":"35c2d7593674a6d9c7cd5cdb0d469916da24075dce9d519ca688058da84ec99b"} Jul 06 22:29:33.093889 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:33.093835 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" podStartSLOduration=0.994101685 podStartE2EDuration="2.093818057s" totalImagesPullingTime="1.099716372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:29:33.091731373 +0000 UTC m=+563.193459035" watchObservedRunningTime="2026-07-06 22:29:33.093818057 +0000 UTC m=+563.195545725" Jul 06 22:29:41.742553 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:41.742512 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:41.743044 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:41.742579 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:41.755153 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:41.755097 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:42.129779 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:42.129752 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-nzrfd" Jul 06 22:29:50.143956 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.143917 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k"] Jul 06 22:29:50.147338 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.147318 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.150268 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.150246 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 22:29:50.164093 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.164059 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k"] Jul 06 22:29:50.213613 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.213573 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.213613 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.213611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.213837 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.213633 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.213837 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.213660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jggt\" (UniqueName: \"kubernetes.io/projected/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-kube-api-access-7jggt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.213837 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.213700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.213837 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.213756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.315041 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.314957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.315041 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.315004 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jggt\" (UniqueName: \"kubernetes.io/projected/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-kube-api-access-7jggt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.315041 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.315030 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.315334 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.315071 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.315334 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.315151 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.315334 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.315171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.315560 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.315537 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.315620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.315585 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.315620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.315614 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.317396 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.317371 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.317570 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.317555 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.326363 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.326339 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jggt\" (UniqueName: \"kubernetes.io/projected/34592bc9-3c31-4bf5-a336-2e4b0a7565c2-kube-api-access-7jggt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k\" (UID: \"34592bc9-3c31-4bf5-a336-2e4b0a7565c2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.457966 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.457926 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:29:50.600215 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:50.600185 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k"] Jul 06 22:29:50.601388 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:29:50.601358 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34592bc9_3c31_4bf5_a336_2e4b0a7565c2.slice/crio-e6282b3c9da486388bdc4a49db2619971a58250c0b0f4efc4b37f16d014c1d06 WatchSource:0}: Error finding container e6282b3c9da486388bdc4a49db2619971a58250c0b0f4efc4b37f16d014c1d06: Status 404 returned error can't find the container with id e6282b3c9da486388bdc4a49db2619971a58250c0b0f4efc4b37f16d014c1d06 Jul 06 22:29:51.169633 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:51.169543 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" event={"ID":"34592bc9-3c31-4bf5-a336-2e4b0a7565c2","Type":"ContainerStarted","Data":"d9dd361e2b5ef50b3730ab3b9b68ea8ae6de0086975afdd50a7003c0383bec8a"} Jul 06 22:29:51.169987 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:51.169648 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" event={"ID":"34592bc9-3c31-4bf5-a336-2e4b0a7565c2","Type":"ContainerStarted","Data":"e6282b3c9da486388bdc4a49db2619971a58250c0b0f4efc4b37f16d014c1d06"} Jul 06 22:29:51.208313 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:29:51.208250 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" podStartSLOduration=1.015628305 podStartE2EDuration="1.208229377s" totalImagesPullingTime="192.601072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:29:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:29:51.205305802 +0000 UTC m=+581.307033464" watchObservedRunningTime="2026-07-06 22:29:51.208229377 +0000 UTC m=+581.309957040" Jul 06 22:30:00.458793 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:00.458748 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:30:00.459188 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:00.458830 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:30:00.472871 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:00.472842 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:30:01.232752 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:01.232724 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k" Jul 06 22:30:02.036038 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.035999 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz"] Jul 06 22:30:02.039628 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.039606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.041895 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.041877 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 22:30:02.051447 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.051417 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz"] Jul 06 22:30:02.130029 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.129987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.130249 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.130042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.130249 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.130077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf2b5\" (UniqueName: \"kubernetes.io/projected/aa4722e2-290d-4501-a223-7edf983f60fe-kube-api-access-qf2b5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.130249 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.130145 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.130249 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.130167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa4722e2-290d-4501-a223-7edf983f60fe-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.130249 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.130189 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.230738 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.230695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa4722e2-290d-4501-a223-7edf983f60fe-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.230738 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.230739 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.230988 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.230768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.230988 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.230791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.230988 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.230821 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf2b5\" (UniqueName: \"kubernetes.io/projected/aa4722e2-290d-4501-a223-7edf983f60fe-kube-api-access-qf2b5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.230988 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.230854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.231224 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.231204 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.231285 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.231265 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.231329 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.231294 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.233137 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.233096 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa4722e2-290d-4501-a223-7edf983f60fe-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.233439 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.233422 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa4722e2-290d-4501-a223-7edf983f60fe-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.241992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.241963 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf2b5\" (UniqueName: \"kubernetes.io/projected/aa4722e2-290d-4501-a223-7edf983f60fe-kube-api-access-qf2b5\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz\" (UID: \"aa4722e2-290d-4501-a223-7edf983f60fe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.350104 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.350068 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:02.497968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:02.497941 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz"] Jul 06 22:30:02.499826 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:30:02.499800 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa4722e2_290d_4501_a223_7edf983f60fe.slice/crio-e8b0dcde13b0634f0b78bad3e6aef9c9b9bd36cf0b01434886afb08f5db0d387 WatchSource:0}: Error finding container e8b0dcde13b0634f0b78bad3e6aef9c9b9bd36cf0b01434886afb08f5db0d387: Status 404 returned error can't find the container with id e8b0dcde13b0634f0b78bad3e6aef9c9b9bd36cf0b01434886afb08f5db0d387 Jul 06 22:30:03.233874 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:03.233831 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" event={"ID":"aa4722e2-290d-4501-a223-7edf983f60fe","Type":"ContainerStarted","Data":"137f9a58a0845562ccd6779095c6213b1eae611ddb9f0b2c8427b583f3678676"} Jul 06 22:30:03.234431 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:03.233889 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" event={"ID":"aa4722e2-290d-4501-a223-7edf983f60fe","Type":"ContainerStarted","Data":"e8b0dcde13b0634f0b78bad3e6aef9c9b9bd36cf0b01434886afb08f5db0d387"} Jul 06 22:30:03.255069 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:03.255014 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" podStartSLOduration=1.033537303 podStartE2EDuration="1.254996433s" totalImagesPullingTime="221.45913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:30:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:30:03.253650399 +0000 UTC m=+593.355378062" watchObservedRunningTime="2026-07-06 22:30:03.254996433 +0000 UTC m=+593.356724123" Jul 06 22:30:12.350873 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:12.350832 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:12.350873 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:12.350886 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:12.363673 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:12.363646 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:30:13.299843 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:30:13.299817 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz" Jul 06 22:31:10.318762 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:10.318728 2578 scope.go:117] "RemoveContainer" containerID="e52c38a23694f206caaa29225df4ef3f02e1af560751d930c938fcc106e4f85d" Jul 06 22:31:18.745005 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:18.744963 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-578bd64fc5-8nsrq"] Jul 06 22:31:18.745408 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:18.745240 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" podUID="e808e501-1581-40dd-9a64-0b228ac751f1" containerName="authorino" containerID="cri-o://04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e" gracePeriod=30 Jul 06 22:31:19.004908 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.004842 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:31:19.027411 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.027380 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp6p8\" (UniqueName: \"kubernetes.io/projected/e808e501-1581-40dd-9a64-0b228ac751f1-kube-api-access-pp6p8\") pod \"e808e501-1581-40dd-9a64-0b228ac751f1\" (UID: \"e808e501-1581-40dd-9a64-0b228ac751f1\") " Jul 06 22:31:19.027593 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.027498 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e808e501-1581-40dd-9a64-0b228ac751f1-tls-cert\") pod \"e808e501-1581-40dd-9a64-0b228ac751f1\" (UID: \"e808e501-1581-40dd-9a64-0b228ac751f1\") " Jul 06 22:31:19.030131 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.030077 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e808e501-1581-40dd-9a64-0b228ac751f1-kube-api-access-pp6p8" (OuterVolumeSpecName: "kube-api-access-pp6p8") pod "e808e501-1581-40dd-9a64-0b228ac751f1" (UID: "e808e501-1581-40dd-9a64-0b228ac751f1"). InnerVolumeSpecName "kube-api-access-pp6p8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:31:19.038766 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.038734 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e808e501-1581-40dd-9a64-0b228ac751f1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e808e501-1581-40dd-9a64-0b228ac751f1" (UID: "e808e501-1581-40dd-9a64-0b228ac751f1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:31:19.128909 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.128861 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e808e501-1581-40dd-9a64-0b228ac751f1-tls-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:31:19.128909 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.128907 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp6p8\" (UniqueName: \"kubernetes.io/projected/e808e501-1581-40dd-9a64-0b228ac751f1-kube-api-access-pp6p8\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:31:19.643501 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.643465 2578 generic.go:356] "Generic (PLEG): container finished" podID="e808e501-1581-40dd-9a64-0b228ac751f1" containerID="04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e" exitCode=0 Jul 06 22:31:19.643671 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.643512 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" Jul 06 22:31:19.643671 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.643547 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" event={"ID":"e808e501-1581-40dd-9a64-0b228ac751f1","Type":"ContainerDied","Data":"04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e"} Jul 06 22:31:19.643671 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.643593 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-578bd64fc5-8nsrq" event={"ID":"e808e501-1581-40dd-9a64-0b228ac751f1","Type":"ContainerDied","Data":"9f0c6fdd33c49a05656d1c87899683dae4f042904b02c885187431ae3fd450c1"} Jul 06 22:31:19.643671 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.643614 2578 scope.go:117] "RemoveContainer" containerID="04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e" Jul 06 22:31:19.652591 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.652573 2578 scope.go:117] "RemoveContainer" containerID="04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e" Jul 06 22:31:19.652885 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:31:19.652853 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e\": container with ID starting with 04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e not found: ID does not exist" containerID="04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e" Jul 06 22:31:19.652939 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.652886 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e"} err="failed to get container status \"04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e\": rpc error: code = NotFound desc = could not find container \"04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e\": container with ID starting with 04e6b709bbd09ca5b01bee3b6e96a005b71f5188e448e9f6dcbbd1d2e051ec9e not found: ID does not exist" Jul 06 22:31:19.690229 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.690199 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-578bd64fc5-8nsrq"] Jul 06 22:31:19.698202 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:19.698170 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-578bd64fc5-8nsrq"] Jul 06 22:31:20.379398 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:31:20.379366 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e808e501-1581-40dd-9a64-0b228ac751f1" path="/var/lib/kubelet/pods/e808e501-1581-40dd-9a64-0b228ac751f1/volumes" Jul 06 22:44:44.108373 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.108336 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk"] Jul 06 22:44:44.109193 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.108899 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e808e501-1581-40dd-9a64-0b228ac751f1" containerName="authorino" Jul 06 22:44:44.109193 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.108920 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e808e501-1581-40dd-9a64-0b228ac751f1" containerName="authorino" Jul 06 22:44:44.109193 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.109008 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e808e501-1581-40dd-9a64-0b228ac751f1" containerName="authorino" Jul 06 22:44:44.111897 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.111872 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.114312 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.114288 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c448c87f-openshift-default-dockercfg-nx67m\"" Jul 06 22:44:44.133922 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.133891 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk"] Jul 06 22:44:44.210855 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.210815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-socket\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.210855 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.210848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istiod-ca-cert\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.211100 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.210886 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-certs\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.211100 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.210902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-envoy\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.211100 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.210959 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-token\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.211100 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.210992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-podinfo\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.211100 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.211016 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-credential-socket\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.211100 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.211050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppgj2\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-kube-api-access-ppgj2\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.211100 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.211068 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-data\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.311806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.311763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-socket\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.311806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.311810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istiod-ca-cert\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312077 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.311844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-certs\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312077 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.311866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-envoy\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312077 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.311908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-token\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312077 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.311936 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-podinfo\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312077 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.311968 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-credential-socket\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312379 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.312356 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-socket\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312429 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.312387 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-credential-socket\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312429 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.312416 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-certs\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312547 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.312442 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppgj2\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-kube-api-access-ppgj2\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.312547 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.312479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-data\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.313102 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.313073 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istiod-ca-cert\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.313372 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.313349 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-data\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.314817 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.314790 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-envoy\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.315030 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.315003 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-podinfo\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.322155 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.322125 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppgj2\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-kube-api-access-ppgj2\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.322268 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.322164 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-token\") pod \"e2e-ait-c448c87f-openshift-default-5d99496469-9prwk\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.423710 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.423612 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:44.564952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.564918 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk"] Jul 06 22:44:44.567988 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:44:44.567957 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4b6cf48_b6fc_4c7c_9abd_dede643b1c17.slice/crio-8a943d0ab3560c218805a0b460ce6e12d0ef1da196a8749e5e240380fe642b32 WatchSource:0}: Error finding container 8a943d0ab3560c218805a0b460ce6e12d0ef1da196a8749e5e240380fe642b32: Status 404 returned error can't find the container with id 8a943d0ab3560c218805a0b460ce6e12d0ef1da196a8749e5e240380fe642b32 Jul 06 22:44:44.570189 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.570157 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:44:44.570273 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.570226 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:44:44.570273 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.570256 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:44:44.887275 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.887231 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" event={"ID":"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17","Type":"ContainerStarted","Data":"2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94"} Jul 06 22:44:44.887462 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.887296 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" event={"ID":"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17","Type":"ContainerStarted","Data":"8a943d0ab3560c218805a0b460ce6e12d0ef1da196a8749e5e240380fe642b32"} Jul 06 22:44:44.907256 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:44.907200 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" podStartSLOduration=0.907184621 podStartE2EDuration="907.184621ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:44:44.905590895 +0000 UTC m=+1475.007318560" watchObservedRunningTime="2026-07-06 22:44:44.907184621 +0000 UTC m=+1475.008912283" Jul 06 22:44:45.423820 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:45.423773 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:45.428768 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:45.428743 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:45.893045 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:45.893012 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:45.894010 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:45.893992 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:50.563095 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:50.563044 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk"] Jul 06 22:44:50.563602 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:50.563298 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" podUID="b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" containerName="istio-proxy" containerID="cri-o://2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94" gracePeriod=30 Jul 06 22:44:55.814778 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.814751 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:55.925705 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.925653 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-token\") pod \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " Jul 06 22:44:55.925705 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.925709 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-envoy\") pod \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " Jul 06 22:44:55.925974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.925760 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-certs\") pod \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " Jul 06 22:44:55.925974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.925787 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-credential-socket\") pod \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " Jul 06 22:44:55.925974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.925809 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-data\") pod \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " Jul 06 22:44:55.925974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.925841 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-podinfo\") pod \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " Jul 06 22:44:55.925974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.925873 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppgj2\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-kube-api-access-ppgj2\") pod \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " Jul 06 22:44:55.925974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.925951 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-socket\") pod \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " Jul 06 22:44:55.926295 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.925981 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istiod-ca-cert\") pod \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\" (UID: \"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17\") " Jul 06 22:44:55.926295 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.926201 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" (UID: "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:44:55.926295 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.926267 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" (UID: "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:44:55.926456 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.926313 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" (UID: "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:44:55.926456 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.926424 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:44:55.926456 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.926443 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:44:55.926598 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.926459 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:44:55.926598 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.926460 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" (UID: "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:44:55.926598 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.926546 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" (UID: "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:44:55.928065 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.928034 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-kube-api-access-ppgj2" (OuterVolumeSpecName: "kube-api-access-ppgj2") pod "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" (UID: "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17"). InnerVolumeSpecName "kube-api-access-ppgj2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:44:55.928206 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.928186 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" (UID: "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:44:55.928343 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.928324 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" (UID: "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:44:55.928402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.928352 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" (UID: "b4b6cf48-b6fc-4c7c-9abd-dede643b1c17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:44:55.945505 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.945472 2578 generic.go:356] "Generic (PLEG): container finished" podID="b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" containerID="2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94" exitCode=0 Jul 06 22:44:55.945644 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.945527 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" Jul 06 22:44:55.945644 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.945550 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" event={"ID":"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17","Type":"ContainerDied","Data":"2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94"} Jul 06 22:44:55.945644 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.945627 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk" event={"ID":"b4b6cf48-b6fc-4c7c-9abd-dede643b1c17","Type":"ContainerDied","Data":"8a943d0ab3560c218805a0b460ce6e12d0ef1da196a8749e5e240380fe642b32"} Jul 06 22:44:55.945747 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.945652 2578 scope.go:117] "RemoveContainer" containerID="2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94" Jul 06 22:44:55.955060 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.955038 2578 scope.go:117] "RemoveContainer" containerID="2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94" Jul 06 22:44:55.955401 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:44:55.955381 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94\": container with ID starting with 2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94 not found: ID does not exist" containerID="2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94" Jul 06 22:44:55.955455 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.955410 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94"} err="failed to get container status \"2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94\": rpc error: code = NotFound desc = could not find container \"2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94\": container with ID starting with 2cba5bfff81574f07d8f2496c07a8436f2673d081c424b94104ce18e2b84eb94 not found: ID does not exist" Jul 06 22:44:55.967867 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.967837 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk"] Jul 06 22:44:55.971902 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:55.971878 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c448c87f-openshift-default-5d99496469-9prwk"] Jul 06 22:44:56.027498 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.027461 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:44:56.027498 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.027492 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppgj2\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-kube-api-access-ppgj2\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:44:56.027498 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.027505 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:44:56.027735 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.027514 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:44:56.027735 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.027523 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:44:56.027735 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.027531 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:44:56.378690 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.378656 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" path="/var/lib/kubelet/pods/b4b6cf48-b6fc-4c7c-9abd-dede643b1c17/volumes" Jul 06 22:44:56.978946 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.978906 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv"] Jul 06 22:44:56.979398 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.979381 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" containerName="istio-proxy" Jul 06 22:44:56.979439 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.979402 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" containerName="istio-proxy" Jul 06 22:44:56.979489 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.979478 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4b6cf48-b6fc-4c7c-9abd-dede643b1c17" containerName="istio-proxy" Jul 06 22:44:56.984007 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.983985 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:56.986606 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.986576 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5980e803-openshift-default-dockercfg-n5cmh\"" Jul 06 22:44:56.995749 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:56.995699 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv"] Jul 06 22:44:57.035308 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.035275 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-envoy\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.035479 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.035324 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-socket\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.035479 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.035375 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-token\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.035479 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.035408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf22b\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-kube-api-access-qf22b\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.035479 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.035449 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/559b7db5-dec0-45e7-90c4-9878cb4b917f-istiod-ca-cert\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.035606 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.035488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-podinfo\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.035606 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.035511 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-credential-socket\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.035606 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.035540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-certs\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.035606 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.035564 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-data\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.136418 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf22b\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-kube-api-access-qf22b\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.136624 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136430 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/559b7db5-dec0-45e7-90c4-9878cb4b917f-istiod-ca-cert\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.136624 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-podinfo\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.136624 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136494 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-credential-socket\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.136624 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-certs\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.136624 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136546 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-data\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.136624 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136575 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-envoy\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.136624 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136613 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-socket\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.136938 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-token\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.137021 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.136988 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-certs\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.137084 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.137067 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-socket\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.137162 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.137133 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-credential-socket\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.137217 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.137187 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-data\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.137338 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.137316 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/559b7db5-dec0-45e7-90c4-9878cb4b917f-istiod-ca-cert\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.139098 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.139069 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-envoy\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.139211 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.139195 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-podinfo\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.145805 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.145772 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf22b\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-kube-api-access-qf22b\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.145805 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.145798 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-token\") pod \"e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.296404 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.296294 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:57.440931 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.440905 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv"] Jul 06 22:44:57.443058 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:44:57.443027 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559b7db5_dec0_45e7_90c4_9878cb4b917f.slice/crio-f10b1eaa74cfd04350a0568834aa0162b19cf27fbe13d2d3de75d560aaad6ea4 WatchSource:0}: Error finding container f10b1eaa74cfd04350a0568834aa0162b19cf27fbe13d2d3de75d560aaad6ea4: Status 404 returned error can't find the container with id f10b1eaa74cfd04350a0568834aa0162b19cf27fbe13d2d3de75d560aaad6ea4 Jul 06 22:44:57.445203 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.445163 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:44:57.445304 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.445232 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:44:57.445304 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.445263 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:44:57.959852 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.959811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" event={"ID":"559b7db5-dec0-45e7-90c4-9878cb4b917f","Type":"ContainerStarted","Data":"400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738"} Jul 06 22:44:57.960053 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.959877 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" event={"ID":"559b7db5-dec0-45e7-90c4-9878cb4b917f","Type":"ContainerStarted","Data":"f10b1eaa74cfd04350a0568834aa0162b19cf27fbe13d2d3de75d560aaad6ea4"} Jul 06 22:44:57.981521 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:57.981459 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" podStartSLOduration=1.981437731 podStartE2EDuration="1.981437731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:44:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:44:57.980633353 +0000 UTC m=+1488.082361016" watchObservedRunningTime="2026-07-06 22:44:57.981437731 +0000 UTC m=+1488.083165394" Jul 06 22:44:58.297373 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:58.297275 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:58.316047 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:58.316005 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:58.964739 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:58.964709 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:44:58.965676 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:44:58.965654 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:45:00.135316 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.135278 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722965-gl249"] Jul 06 22:45:00.138883 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.138864 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6"] Jul 06 22:45:00.138999 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.138978 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-gl249" Jul 06 22:45:00.141667 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.141639 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-98x7w\"" Jul 06 22:45:00.142316 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.142296 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" Jul 06 22:45:00.147812 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.147742 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6"] Jul 06 22:45:00.151621 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.151597 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722965-gl249"] Jul 06 22:45:00.267599 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.267557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d9rf\" (UniqueName: \"kubernetes.io/projected/cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88-kube-api-access-6d9rf\") pod \"maas-api-key-cleanup-29722965-gl249\" (UID: \"cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88\") " pod="opendatahub/maas-api-key-cleanup-29722965-gl249" Jul 06 22:45:00.267798 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.267620 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6prvp\" (UniqueName: \"kubernetes.io/projected/9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1-kube-api-access-6prvp\") pod \"maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6\" (UID: \"9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1\") " pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" Jul 06 22:45:00.368591 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.368548 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d9rf\" (UniqueName: \"kubernetes.io/projected/cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88-kube-api-access-6d9rf\") pod \"maas-api-key-cleanup-29722965-gl249\" (UID: \"cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88\") " pod="opendatahub/maas-api-key-cleanup-29722965-gl249" Jul 06 22:45:00.368771 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.368605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6prvp\" (UniqueName: \"kubernetes.io/projected/9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1-kube-api-access-6prvp\") pod \"maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6\" (UID: \"9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1\") " pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" Jul 06 22:45:00.377801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.377764 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d9rf\" (UniqueName: \"kubernetes.io/projected/cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88-kube-api-access-6d9rf\") pod \"maas-api-key-cleanup-29722965-gl249\" (UID: \"cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88\") " pod="opendatahub/maas-api-key-cleanup-29722965-gl249" Jul 06 22:45:00.377934 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.377844 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6prvp\" (UniqueName: \"kubernetes.io/projected/9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1-kube-api-access-6prvp\") pod \"maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6\" (UID: \"9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1\") " pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" Jul 06 22:45:00.457819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.457725 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-gl249" Jul 06 22:45:00.465930 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.465902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" Jul 06 22:45:00.618142 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.618084 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722965-gl249"] Jul 06 22:45:00.619878 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:45:00.619851 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfa4cae5_b6e3_4dd9_bd8d_9c0f748c6b88.slice/crio-bd5f2d9aca310e1b289db6127c5d48a1fa295ea8d60e6bdf4049ac1c3e082ccd WatchSource:0}: Error finding container bd5f2d9aca310e1b289db6127c5d48a1fa295ea8d60e6bdf4049ac1c3e082ccd: Status 404 returned error can't find the container with id bd5f2d9aca310e1b289db6127c5d48a1fa295ea8d60e6bdf4049ac1c3e082ccd Jul 06 22:45:00.622140 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.622104 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:45:00.643273 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.643249 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6"] Jul 06 22:45:00.644964 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:45:00.644939 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd63d3e_ce49_43d9_a8c6_1f4326b43ff1.slice/crio-0a20fd5b192e902b7906520b8b11373321f37317ddf3bb3a8ac06ea4937a089b WatchSource:0}: Error finding container 0a20fd5b192e902b7906520b8b11373321f37317ddf3bb3a8ac06ea4937a089b: Status 404 returned error can't find the container with id 0a20fd5b192e902b7906520b8b11373321f37317ddf3bb3a8ac06ea4937a089b Jul 06 22:45:00.982187 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.982151 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" event={"ID":"9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1","Type":"ContainerStarted","Data":"0a20fd5b192e902b7906520b8b11373321f37317ddf3bb3a8ac06ea4937a089b"} Jul 06 22:45:00.983336 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:00.983304 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-gl249" event={"ID":"cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88","Type":"ContainerStarted","Data":"bd5f2d9aca310e1b289db6127c5d48a1fa295ea8d60e6bdf4049ac1c3e082ccd"} Jul 06 22:45:03.661780 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:03.661733 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6"] Jul 06 22:45:04.007466 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:04.007366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" event={"ID":"9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1","Type":"ContainerStarted","Data":"2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d"} Jul 06 22:45:04.007466 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:04.007416 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" podUID="9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1" containerName="cleanup" containerID="cri-o://2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d" gracePeriod=30 Jul 06 22:45:04.008988 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:04.008958 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-gl249" event={"ID":"cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88","Type":"ContainerStarted","Data":"fe1d9fb7d601b79bc8aa802aff7ae0e5a57121549a76ac9463e708931875aab0"} Jul 06 22:45:04.024023 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:04.023973 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" podStartSLOduration=1.2372189 podStartE2EDuration="4.023957418s" totalImagesPullingTime="2.786738518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:45:04.023488852 +0000 UTC m=+1494.125216516" watchObservedRunningTime="2026-07-06 22:45:04.023957418 +0000 UTC m=+1494.125685072" Jul 06 22:45:04.039803 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:04.039748 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722965-gl249" podStartSLOduration=1.223163787 podStartE2EDuration="4.039733914s" totalImagesPullingTime="2.816570127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:45:04.038308101 +0000 UTC m=+1494.140035763" watchObservedRunningTime="2026-07-06 22:45:04.039733914 +0000 UTC m=+1494.141461587" Jul 06 22:45:04.697252 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:04.697190 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" Jul 06 22:45:04.812234 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:04.812202 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6prvp\" (UniqueName: \"kubernetes.io/projected/9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1-kube-api-access-6prvp\") pod \"9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1\" (UID: \"9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1\") " Jul 06 22:45:04.814267 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:04.814239 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1-kube-api-access-6prvp" (OuterVolumeSpecName: "kube-api-access-6prvp") pod "9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1" (UID: "9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1"). InnerVolumeSpecName "kube-api-access-6prvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:45:04.913676 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:04.913645 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6prvp\" (UniqueName: \"kubernetes.io/projected/9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1-kube-api-access-6prvp\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:05.015538 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.015500 2578 generic.go:356] "Generic (PLEG): container finished" podID="9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1" containerID="2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d" exitCode=6 Jul 06 22:45:05.015538 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.015541 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" Jul 06 22:45:05.015767 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.015581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" event={"ID":"9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1","Type":"ContainerDied","Data":"2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d"} Jul 06 22:45:05.015767 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.015639 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6" event={"ID":"9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1","Type":"ContainerDied","Data":"0a20fd5b192e902b7906520b8b11373321f37317ddf3bb3a8ac06ea4937a089b"} Jul 06 22:45:05.015767 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.015664 2578 scope.go:117] "RemoveContainer" containerID="2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d" Jul 06 22:45:05.017269 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.017241 2578 generic.go:356] "Generic (PLEG): container finished" podID="cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88" containerID="fe1d9fb7d601b79bc8aa802aff7ae0e5a57121549a76ac9463e708931875aab0" exitCode=0 Jul 06 22:45:05.017382 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.017326 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-gl249" event={"ID":"cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88","Type":"ContainerDied","Data":"fe1d9fb7d601b79bc8aa802aff7ae0e5a57121549a76ac9463e708931875aab0"} Jul 06 22:45:05.025569 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.025550 2578 scope.go:117] "RemoveContainer" containerID="2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d" Jul 06 22:45:05.025787 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:45:05.025770 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d\": container with ID starting with 2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d not found: ID does not exist" containerID="2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d" Jul 06 22:45:05.025835 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.025795 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d"} err="failed to get container status \"2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d\": rpc error: code = NotFound desc = could not find container \"2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d\": container with ID starting with 2a13e2879e34d80349f74e1432645901d1185b641ad5acb522bc99ed42c91c0d not found: ID does not exist" Jul 06 22:45:05.077940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.077831 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6"] Jul 06 22:45:05.082773 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:05.082735 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-ait-5980e803-29722965-thsm6"] Jul 06 22:45:06.161748 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:06.161719 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-gl249" Jul 06 22:45:06.223449 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:06.223417 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d9rf\" (UniqueName: \"kubernetes.io/projected/cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88-kube-api-access-6d9rf\") pod \"cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88\" (UID: \"cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88\") " Jul 06 22:45:06.225443 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:06.225411 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88-kube-api-access-6d9rf" (OuterVolumeSpecName: "kube-api-access-6d9rf") pod "cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88" (UID: "cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88"). InnerVolumeSpecName "kube-api-access-6d9rf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:45:06.324744 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:06.324701 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d9rf\" (UniqueName: \"kubernetes.io/projected/cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88-kube-api-access-6d9rf\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:06.379777 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:06.379741 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1" path="/var/lib/kubelet/pods/9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1/volumes" Jul 06 22:45:07.031177 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:07.031150 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-gl249" Jul 06 22:45:07.031360 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:07.031143 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-gl249" event={"ID":"cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88","Type":"ContainerDied","Data":"bd5f2d9aca310e1b289db6127c5d48a1fa295ea8d60e6bdf4049ac1c3e082ccd"} Jul 06 22:45:07.031360 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:07.031295 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd5f2d9aca310e1b289db6127c5d48a1fa295ea8d60e6bdf4049ac1c3e082ccd" Jul 06 22:45:09.043493 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:09.043458 2578 generic.go:356] "Generic (PLEG): container finished" podID="559b7db5-dec0-45e7-90c4-9878cb4b917f" containerID="400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738" exitCode=0 Jul 06 22:45:09.043937 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:09.043530 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" event={"ID":"559b7db5-dec0-45e7-90c4-9878cb4b917f","Type":"ContainerDied","Data":"400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738"} Jul 06 22:45:09.043937 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:09.043929 2578 scope.go:117] "RemoveContainer" containerID="400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738" Jul 06 22:45:09.045015 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:09.044987 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:45:09.045089 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:09.045072 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:45:09.045153 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:09.045139 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:45:09.320068 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:09.319981 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv"] Jul 06 22:45:10.050900 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:10.050862 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" event={"ID":"559b7db5-dec0-45e7-90c4-9878cb4b917f","Type":"ContainerStarted","Data":"a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1"} Jul 06 22:45:10.051350 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:10.050931 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" podUID="559b7db5-dec0-45e7-90c4-9878cb4b917f" containerName="istio-proxy" containerID="cri-o://a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1" gracePeriod=30 Jul 06 22:45:13.964842 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:13.964805 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:45:15.313626 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.313559 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:45:15.404571 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404521 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-token\") pod \"559b7db5-dec0-45e7-90c4-9878cb4b917f\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " Jul 06 22:45:15.404752 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404598 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-podinfo\") pod \"559b7db5-dec0-45e7-90c4-9878cb4b917f\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " Jul 06 22:45:15.404752 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404622 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-credential-socket\") pod \"559b7db5-dec0-45e7-90c4-9878cb4b917f\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " Jul 06 22:45:15.404752 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404653 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-certs\") pod \"559b7db5-dec0-45e7-90c4-9878cb4b917f\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " Jul 06 22:45:15.404752 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404699 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-data\") pod \"559b7db5-dec0-45e7-90c4-9878cb4b917f\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " Jul 06 22:45:15.404752 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404728 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qf22b\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-kube-api-access-qf22b\") pod \"559b7db5-dec0-45e7-90c4-9878cb4b917f\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " Jul 06 22:45:15.404998 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404760 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-envoy\") pod \"559b7db5-dec0-45e7-90c4-9878cb4b917f\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " Jul 06 22:45:15.404998 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404805 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-socket\") pod \"559b7db5-dec0-45e7-90c4-9878cb4b917f\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " Jul 06 22:45:15.404998 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404850 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/559b7db5-dec0-45e7-90c4-9878cb4b917f-istiod-ca-cert\") pod \"559b7db5-dec0-45e7-90c4-9878cb4b917f\" (UID: \"559b7db5-dec0-45e7-90c4-9878cb4b917f\") " Jul 06 22:45:15.404998 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404928 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "559b7db5-dec0-45e7-90c4-9878cb4b917f" (UID: "559b7db5-dec0-45e7-90c4-9878cb4b917f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:15.404998 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.404968 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "559b7db5-dec0-45e7-90c4-9878cb4b917f" (UID: "559b7db5-dec0-45e7-90c4-9878cb4b917f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:15.405298 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.405181 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "559b7db5-dec0-45e7-90c4-9878cb4b917f" (UID: "559b7db5-dec0-45e7-90c4-9878cb4b917f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:15.405298 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.405194 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:15.405298 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.405214 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:15.405298 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.405271 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/559b7db5-dec0-45e7-90c4-9878cb4b917f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "559b7db5-dec0-45e7-90c4-9878cb4b917f" (UID: "559b7db5-dec0-45e7-90c4-9878cb4b917f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:45:15.405713 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.405685 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-data" (OuterVolumeSpecName: "istio-data") pod "559b7db5-dec0-45e7-90c4-9878cb4b917f" (UID: "559b7db5-dec0-45e7-90c4-9878cb4b917f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:15.406896 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.406873 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "559b7db5-dec0-45e7-90c4-9878cb4b917f" (UID: "559b7db5-dec0-45e7-90c4-9878cb4b917f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:45:15.407214 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.407188 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-kube-api-access-qf22b" (OuterVolumeSpecName: "kube-api-access-qf22b") pod "559b7db5-dec0-45e7-90c4-9878cb4b917f" (UID: "559b7db5-dec0-45e7-90c4-9878cb4b917f"). InnerVolumeSpecName "kube-api-access-qf22b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:45:15.407214 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.407205 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-token" (OuterVolumeSpecName: "istio-token") pod "559b7db5-dec0-45e7-90c4-9878cb4b917f" (UID: "559b7db5-dec0-45e7-90c4-9878cb4b917f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:45:15.407335 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.407188 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "559b7db5-dec0-45e7-90c4-9878cb4b917f" (UID: "559b7db5-dec0-45e7-90c4-9878cb4b917f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:15.506570 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.506534 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:15.506570 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.506564 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf22b\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-kube-api-access-qf22b\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:15.506570 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.506574 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:15.506801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.506585 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/559b7db5-dec0-45e7-90c4-9878cb4b917f-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:15.506801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.506593 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/559b7db5-dec0-45e7-90c4-9878cb4b917f-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:15.506801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.506603 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:15.506801 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.506610 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/559b7db5-dec0-45e7-90c4-9878cb4b917f-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:15.984802 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.984757 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc"] Jul 06 22:45:15.985323 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985301 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="559b7db5-dec0-45e7-90c4-9878cb4b917f" containerName="istio-proxy" Jul 06 22:45:15.985323 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985323 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="559b7db5-dec0-45e7-90c4-9878cb4b917f" containerName="istio-proxy" Jul 06 22:45:15.985486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985342 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1" containerName="cleanup" Jul 06 22:45:15.985486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985352 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1" containerName="cleanup" Jul 06 22:45:15.985486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985367 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88" containerName="cleanup" Jul 06 22:45:15.985486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985376 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88" containerName="cleanup" Jul 06 22:45:15.985486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985387 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="559b7db5-dec0-45e7-90c4-9878cb4b917f" containerName="istio-proxy" Jul 06 22:45:15.985486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985395 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="559b7db5-dec0-45e7-90c4-9878cb4b917f" containerName="istio-proxy" Jul 06 22:45:15.985694 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985506 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="559b7db5-dec0-45e7-90c4-9878cb4b917f" containerName="istio-proxy" Jul 06 22:45:15.985694 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985517 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88" containerName="cleanup" Jul 06 22:45:15.985694 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.985524 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd63d3e-ce49-43d9-a8c6-1f4326b43ff1" containerName="cleanup" Jul 06 22:45:15.988829 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.988810 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:15.990986 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:15.990966 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-3f0ec651-openshift-default-dockercfg-69xxd\"" Jul 06 22:45:16.001133 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.001080 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc"] Jul 06 22:45:16.092427 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.092392 2578 generic.go:356] "Generic (PLEG): container finished" podID="559b7db5-dec0-45e7-90c4-9878cb4b917f" containerID="a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1" exitCode=0 Jul 06 22:45:16.092591 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.092459 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" Jul 06 22:45:16.092591 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.092497 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" event={"ID":"559b7db5-dec0-45e7-90c4-9878cb4b917f","Type":"ContainerDied","Data":"a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1"} Jul 06 22:45:16.092673 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.092602 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv" event={"ID":"559b7db5-dec0-45e7-90c4-9878cb4b917f","Type":"ContainerDied","Data":"f10b1eaa74cfd04350a0568834aa0162b19cf27fbe13d2d3de75d560aaad6ea4"} Jul 06 22:45:16.092673 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.092638 2578 scope.go:117] "RemoveContainer" containerID="a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1" Jul 06 22:45:16.104380 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.104337 2578 scope.go:117] "RemoveContainer" containerID="400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738" Jul 06 22:45:16.112266 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.112234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-credential-socket\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.112403 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.112288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-token\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.112403 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.112362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n64j\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-kube-api-access-4n64j\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.112403 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.112393 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-envoy\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.112555 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.112415 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-certs\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.112555 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.112500 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-podinfo\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.112555 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.112535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-data\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.112555 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.112553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c24199-c6d5-4277-bcff-c2542628d9fb-istiod-ca-cert\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.112740 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.112602 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-socket\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.114784 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.114687 2578 scope.go:117] "RemoveContainer" containerID="a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1" Jul 06 22:45:16.115037 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:45:16.115014 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1\": container with ID starting with a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1 not found: ID does not exist" containerID="a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1" Jul 06 22:45:16.115078 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.115049 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1"} err="failed to get container status \"a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1\": rpc error: code = NotFound desc = could not find container \"a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1\": container with ID starting with a5ea968ad6910dc4edd540b0108edcac9207d69d6aabca2ce22db7b471ce85e1 not found: ID does not exist" Jul 06 22:45:16.115078 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.115072 2578 scope.go:117] "RemoveContainer" containerID="400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738" Jul 06 22:45:16.115442 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:45:16.115424 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738\": container with ID starting with 400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738 not found: ID does not exist" containerID="400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738" Jul 06 22:45:16.115478 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.115448 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738"} err="failed to get container status \"400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738\": rpc error: code = NotFound desc = could not find container \"400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738\": container with ID starting with 400c681310669d9beb51af824fb663182c7451f673dfef50628ce107b5828738 not found: ID does not exist" Jul 06 22:45:16.117275 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.117249 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv"] Jul 06 22:45:16.121731 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.121696 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5980e803-openshift-default-75ccbc9bd7-b6whv"] Jul 06 22:45:16.213088 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-podinfo\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213319 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-data\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213319 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213155 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c24199-c6d5-4277-bcff-c2542628d9fb-istiod-ca-cert\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213319 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213192 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-socket\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213319 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213248 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-credential-socket\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213319 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213273 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-token\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213577 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213320 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n64j\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-kube-api-access-4n64j\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213577 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213351 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-envoy\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213577 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213383 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-certs\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213738 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213651 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-data\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.213912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213880 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-credential-socket\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.214021 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213913 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-socket\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.214021 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213924 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c24199-c6d5-4277-bcff-c2542628d9fb-istiod-ca-cert\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.214021 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.213949 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-certs\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.215749 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.215731 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-podinfo\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.215852 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.215834 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-envoy\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.223664 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.223639 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-token\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.223876 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.223852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n64j\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-kube-api-access-4n64j\") pod \"e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.300178 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.300072 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:16.380973 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.380928 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="559b7db5-dec0-45e7-90c4-9878cb4b917f" path="/var/lib/kubelet/pods/559b7db5-dec0-45e7-90c4-9878cb4b917f/volumes" Jul 06 22:45:16.457137 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.457089 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc"] Jul 06 22:45:16.459921 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:45:16.459893 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c24199_c6d5_4277_bcff_c2542628d9fb.slice/crio-8ddf63a508b5760e93658e66bf3dfcb808fa443d64b65f26f1a0bcaed9ea3566 WatchSource:0}: Error finding container 8ddf63a508b5760e93658e66bf3dfcb808fa443d64b65f26f1a0bcaed9ea3566: Status 404 returned error can't find the container with id 8ddf63a508b5760e93658e66bf3dfcb808fa443d64b65f26f1a0bcaed9ea3566 Jul 06 22:45:16.462083 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.462051 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:45:16.462262 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.462217 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:45:16.462310 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:16.462294 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:45:17.102158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:17.101994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" event={"ID":"40c24199-c6d5-4277-bcff-c2542628d9fb","Type":"ContainerStarted","Data":"2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2"} Jul 06 22:45:17.102158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:17.102064 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" event={"ID":"40c24199-c6d5-4277-bcff-c2542628d9fb","Type":"ContainerStarted","Data":"8ddf63a508b5760e93658e66bf3dfcb808fa443d64b65f26f1a0bcaed9ea3566"} Jul 06 22:45:17.125750 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:17.125693 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" podStartSLOduration=2.125676254 podStartE2EDuration="2.125676254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:45:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:45:17.123073537 +0000 UTC m=+1507.224801199" watchObservedRunningTime="2026-07-06 22:45:17.125676254 +0000 UTC m=+1507.227403978" Jul 06 22:45:17.300996 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:17.300960 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:17.302543 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:17.302514 2578 patch_prober.go:28] interesting pod/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 06 22:45:17.302655 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:17.302577 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" podUID="40c24199-c6d5-4277-bcff-c2542628d9fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 06 22:45:18.304942 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:18.304910 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:19.113650 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:19.113617 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:19.114629 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:19.114610 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:21.761678 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:21.761642 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc"] Jul 06 22:45:22.131390 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:22.131349 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" podUID="40c24199-c6d5-4277-bcff-c2542628d9fb" containerName="istio-proxy" containerID="cri-o://2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2" gracePeriod=30 Jul 06 22:45:27.393379 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.393317 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:27.530362 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530321 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-envoy\") pod \"40c24199-c6d5-4277-bcff-c2542628d9fb\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " Jul 06 22:45:27.530539 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530361 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c24199-c6d5-4277-bcff-c2542628d9fb-istiod-ca-cert\") pod \"40c24199-c6d5-4277-bcff-c2542628d9fb\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " Jul 06 22:45:27.530539 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530422 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-podinfo\") pod \"40c24199-c6d5-4277-bcff-c2542628d9fb\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " Jul 06 22:45:27.530539 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530440 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-socket\") pod \"40c24199-c6d5-4277-bcff-c2542628d9fb\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " Jul 06 22:45:27.530539 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530470 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-data\") pod \"40c24199-c6d5-4277-bcff-c2542628d9fb\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " Jul 06 22:45:27.530539 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530513 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-token\") pod \"40c24199-c6d5-4277-bcff-c2542628d9fb\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " Jul 06 22:45:27.530797 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530541 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-credential-socket\") pod \"40c24199-c6d5-4277-bcff-c2542628d9fb\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " Jul 06 22:45:27.530797 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530577 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n64j\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-kube-api-access-4n64j\") pod \"40c24199-c6d5-4277-bcff-c2542628d9fb\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " Jul 06 22:45:27.530797 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530602 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-certs\") pod \"40c24199-c6d5-4277-bcff-c2542628d9fb\" (UID: \"40c24199-c6d5-4277-bcff-c2542628d9fb\") " Jul 06 22:45:27.530960 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530794 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40c24199-c6d5-4277-bcff-c2542628d9fb" (UID: "40c24199-c6d5-4277-bcff-c2542628d9fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:27.530960 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530879 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c24199-c6d5-4277-bcff-c2542628d9fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40c24199-c6d5-4277-bcff-c2542628d9fb" (UID: "40c24199-c6d5-4277-bcff-c2542628d9fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:45:27.531071 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.530963 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:27.531071 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.531002 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40c24199-c6d5-4277-bcff-c2542628d9fb" (UID: "40c24199-c6d5-4277-bcff-c2542628d9fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:27.531210 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.531143 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40c24199-c6d5-4277-bcff-c2542628d9fb" (UID: "40c24199-c6d5-4277-bcff-c2542628d9fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:27.531806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.531765 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "40c24199-c6d5-4277-bcff-c2542628d9fb" (UID: "40c24199-c6d5-4277-bcff-c2542628d9fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:27.532899 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.532878 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40c24199-c6d5-4277-bcff-c2542628d9fb" (UID: "40c24199-c6d5-4277-bcff-c2542628d9fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:45:27.532984 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.532935 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-kube-api-access-4n64j" (OuterVolumeSpecName: "kube-api-access-4n64j") pod "40c24199-c6d5-4277-bcff-c2542628d9fb" (UID: "40c24199-c6d5-4277-bcff-c2542628d9fb"). InnerVolumeSpecName "kube-api-access-4n64j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:45:27.533078 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.533061 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40c24199-c6d5-4277-bcff-c2542628d9fb" (UID: "40c24199-c6d5-4277-bcff-c2542628d9fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:45:27.533148 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.533081 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "40c24199-c6d5-4277-bcff-c2542628d9fb" (UID: "40c24199-c6d5-4277-bcff-c2542628d9fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:45:27.632320 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.632282 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:27.632320 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.632311 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:27.632320 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.632320 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:27.632320 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.632330 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n64j\" (UniqueName: \"kubernetes.io/projected/40c24199-c6d5-4277-bcff-c2542628d9fb-kube-api-access-4n64j\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:27.632591 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.632339 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:27.632591 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.632347 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:27.632591 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.632355 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c24199-c6d5-4277-bcff-c2542628d9fb-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:27.632591 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:27.632364 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c24199-c6d5-4277-bcff-c2542628d9fb-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:45:28.164855 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.164818 2578 generic.go:356] "Generic (PLEG): container finished" podID="40c24199-c6d5-4277-bcff-c2542628d9fb" containerID="2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2" exitCode=0 Jul 06 22:45:28.165029 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.164866 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" Jul 06 22:45:28.165029 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.164908 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" event={"ID":"40c24199-c6d5-4277-bcff-c2542628d9fb","Type":"ContainerDied","Data":"2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2"} Jul 06 22:45:28.165029 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.164965 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc" event={"ID":"40c24199-c6d5-4277-bcff-c2542628d9fb","Type":"ContainerDied","Data":"8ddf63a508b5760e93658e66bf3dfcb808fa443d64b65f26f1a0bcaed9ea3566"} Jul 06 22:45:28.165029 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.164994 2578 scope.go:117] "RemoveContainer" containerID="2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2" Jul 06 22:45:28.181076 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.181053 2578 scope.go:117] "RemoveContainer" containerID="2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2" Jul 06 22:45:28.181465 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:45:28.181440 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2\": container with ID starting with 2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2 not found: ID does not exist" containerID="2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2" Jul 06 22:45:28.181554 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.181473 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2"} err="failed to get container status \"2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2\": rpc error: code = NotFound desc = could not find container \"2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2\": container with ID starting with 2b6ca88265c73c473163d3c8083330456c38e0d12f18677e01fe01d3a78bd1c2 not found: ID does not exist" Jul 06 22:45:28.193335 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.193307 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc"] Jul 06 22:45:28.198181 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.198157 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-3f0ec651-openshift-default-fbbbc9f5f-h87wc"] Jul 06 22:45:28.378756 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:45:28.378721 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c24199-c6d5-4277-bcff-c2542628d9fb" path="/var/lib/kubelet/pods/40c24199-c6d5-4277-bcff-c2542628d9fb/volumes" Jul 06 22:47:05.281661 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.281565 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr"] Jul 06 22:47:05.282158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.281994 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c24199-c6d5-4277-bcff-c2542628d9fb" containerName="istio-proxy" Jul 06 22:47:05.282158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.282007 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c24199-c6d5-4277-bcff-c2542628d9fb" containerName="istio-proxy" Jul 06 22:47:05.282158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.282084 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="559b7db5-dec0-45e7-90c4-9878cb4b917f" containerName="istio-proxy" Jul 06 22:47:05.282158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.282094 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c24199-c6d5-4277-bcff-c2542628d9fb" containerName="istio-proxy" Jul 06 22:47:05.285138 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.285106 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.287484 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.287464 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-29698a60-openshift-default-dockercfg-l5gth\"" Jul 06 22:47:05.306029 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.306000 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr"] Jul 06 22:47:05.377503 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.377471 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5g8c\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-kube-api-access-m5g8c\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.377727 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.377514 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-socket\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.377727 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.377612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-data\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.377727 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.377667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-podinfo\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.377866 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.377741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-token\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.377866 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.377776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-credential-socket\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.377866 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.377799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-envoy\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.377866 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.377856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istiod-ca-cert\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.377997 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.377919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-certs\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479153 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-data\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479153 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479152 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-podinfo\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479407 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479173 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-token\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479407 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-credential-socket\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479407 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-envoy\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479407 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479228 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istiod-ca-cert\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479407 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-certs\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479671 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479460 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5g8c\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-kube-api-access-m5g8c\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479671 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-socket\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479671 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479572 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-data\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479671 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479605 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-credential-socket\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479853 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479808 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-certs\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.479853 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479826 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-socket\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.480008 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.479988 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istiod-ca-cert\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.481569 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.481548 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-envoy\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.481780 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.481765 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-podinfo\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.487849 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.487828 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5g8c\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-kube-api-access-m5g8c\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.487952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.487832 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-token\") pod \"e2e-mt-29698a60-openshift-default-65795557bc-kx4lr\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.595217 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.595179 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:05.734664 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.734635 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr"] Jul 06 22:47:05.736015 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:47:05.735983 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ecdc025_de93_4732_b2a5_aba7bffb65e4.slice/crio-9c5ec06ec58b32ccf3621f7fbef7a9efc64cca9676bf7dab90726b9c604e9cdc WatchSource:0}: Error finding container 9c5ec06ec58b32ccf3621f7fbef7a9efc64cca9676bf7dab90726b9c604e9cdc: Status 404 returned error can't find the container with id 9c5ec06ec58b32ccf3621f7fbef7a9efc64cca9676bf7dab90726b9c604e9cdc Jul 06 22:47:05.738790 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.738756 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:47:05.738947 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.738840 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:47:05.738947 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:05.738891 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:47:06.698540 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:06.698495 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" event={"ID":"3ecdc025-de93-4732-b2a5-aba7bffb65e4","Type":"ContainerStarted","Data":"176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f"} Jul 06 22:47:06.699277 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:06.698601 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" event={"ID":"3ecdc025-de93-4732-b2a5-aba7bffb65e4","Type":"ContainerStarted","Data":"9c5ec06ec58b32ccf3621f7fbef7a9efc64cca9676bf7dab90726b9c604e9cdc"} Jul 06 22:47:06.718889 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:06.718834 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" podStartSLOduration=1.7188171589999999 podStartE2EDuration="1.718817159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:06.716676696 +0000 UTC m=+1616.818404358" watchObservedRunningTime="2026-07-06 22:47:06.718817159 +0000 UTC m=+1616.820544879" Jul 06 22:47:07.596180 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:07.596139 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:07.601007 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:07.600979 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:07.704612 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:07.704583 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:07.705705 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:07.705686 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:23.217697 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:23.216969 2578 patch_prober.go:28] interesting pod/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": read tcp 10.134.0.2:58196->10.134.0.53:15021: read: connection reset by peer" start-of-body= Jul 06 22:47:23.217697 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:23.217047 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" podUID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": read tcp 10.134.0.2:58196->10.134.0.53:15021: read: connection reset by peer" Jul 06 22:47:23.794653 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:23.794615 2578 generic.go:356] "Generic (PLEG): container finished" podID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerID="176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f" exitCode=0 Jul 06 22:47:23.794802 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:23.794689 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" event={"ID":"3ecdc025-de93-4732-b2a5-aba7bffb65e4","Type":"ContainerDied","Data":"176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f"} Jul 06 22:47:23.795127 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:23.795097 2578 scope.go:117] "RemoveContainer" containerID="176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f" Jul 06 22:47:23.796078 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:23.796053 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:47:23.796154 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:23.796139 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:47:23.796195 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:23.796177 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:47:24.803381 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:24.803334 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" event={"ID":"3ecdc025-de93-4732-b2a5-aba7bffb65e4","Type":"ContainerStarted","Data":"78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f"} Jul 06 22:47:25.596108 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:25.596063 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:25.599614 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:25.599585 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:25.811746 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:25.811712 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:25.812811 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:25.812790 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:27.608098 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:27.608063 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr"] Jul 06 22:47:27.822308 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:27.822271 2578 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" secret="" err="secret \"e2e-mt-29698a60-openshift-default-dockercfg-l5gth\" not found" Jul 06 22:47:28.827181 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:28.827129 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" podUID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerName="istio-proxy" containerID="cri-o://78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f" gracePeriod=30 Jul 06 22:47:34.082062 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.082035 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:34.146702 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146605 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-credential-socket\") pod \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " Jul 06 22:47:34.146702 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146658 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-token\") pod \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " Jul 06 22:47:34.146940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146716 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-data\") pod \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " Jul 06 22:47:34.146940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146739 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-certs\") pod \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " Jul 06 22:47:34.146940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146769 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5g8c\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-kube-api-access-m5g8c\") pod \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " Jul 06 22:47:34.146940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146790 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-socket\") pod \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " Jul 06 22:47:34.146940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146843 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istiod-ca-cert\") pod \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " Jul 06 22:47:34.146940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146870 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-envoy\") pod \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " Jul 06 22:47:34.146940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146907 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-podinfo\") pod \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\" (UID: \"3ecdc025-de93-4732-b2a5-aba7bffb65e4\") " Jul 06 22:47:34.146940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.146909 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ecdc025-de93-4732-b2a5-aba7bffb65e4" (UID: "3ecdc025-de93-4732-b2a5-aba7bffb65e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:47:34.147372 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.147075 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ecdc025-de93-4732-b2a5-aba7bffb65e4" (UID: "3ecdc025-de93-4732-b2a5-aba7bffb65e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:47:34.147372 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.147222 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ecdc025-de93-4732-b2a5-aba7bffb65e4" (UID: "3ecdc025-de93-4732-b2a5-aba7bffb65e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:47:34.147372 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.147249 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:47:34.147372 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.147268 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:47:34.147372 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.147325 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ecdc025-de93-4732-b2a5-aba7bffb65e4" (UID: "3ecdc025-de93-4732-b2a5-aba7bffb65e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:47:34.147640 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.147556 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ecdc025-de93-4732-b2a5-aba7bffb65e4" (UID: "3ecdc025-de93-4732-b2a5-aba7bffb65e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:47:34.149102 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.149078 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ecdc025-de93-4732-b2a5-aba7bffb65e4" (UID: "3ecdc025-de93-4732-b2a5-aba7bffb65e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:47:34.149375 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.149357 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ecdc025-de93-4732-b2a5-aba7bffb65e4" (UID: "3ecdc025-de93-4732-b2a5-aba7bffb65e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:47:34.149430 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.149361 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ecdc025-de93-4732-b2a5-aba7bffb65e4" (UID: "3ecdc025-de93-4732-b2a5-aba7bffb65e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:47:34.149538 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.149520 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-kube-api-access-m5g8c" (OuterVolumeSpecName: "kube-api-access-m5g8c") pod "3ecdc025-de93-4732-b2a5-aba7bffb65e4" (UID: "3ecdc025-de93-4732-b2a5-aba7bffb65e4"). InnerVolumeSpecName "kube-api-access-m5g8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:47:34.247626 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.247597 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:47:34.247626 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.247622 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5g8c\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-kube-api-access-m5g8c\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:47:34.247626 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.247633 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:47:34.247863 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.247642 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:47:34.247863 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.247649 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:47:34.247863 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.247658 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:47:34.247863 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.247665 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ecdc025-de93-4732-b2a5-aba7bffb65e4-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:47:34.859449 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.859415 2578 generic.go:356] "Generic (PLEG): container finished" podID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerID="78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f" exitCode=0 Jul 06 22:47:34.859662 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.859465 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" Jul 06 22:47:34.859662 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.859509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" event={"ID":"3ecdc025-de93-4732-b2a5-aba7bffb65e4","Type":"ContainerDied","Data":"78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f"} Jul 06 22:47:34.859662 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.859567 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr" event={"ID":"3ecdc025-de93-4732-b2a5-aba7bffb65e4","Type":"ContainerDied","Data":"9c5ec06ec58b32ccf3621f7fbef7a9efc64cca9676bf7dab90726b9c604e9cdc"} Jul 06 22:47:34.859662 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.859597 2578 scope.go:117] "RemoveContainer" containerID="78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f" Jul 06 22:47:34.868904 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.868888 2578 scope.go:117] "RemoveContainer" containerID="176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f" Jul 06 22:47:34.877485 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.877457 2578 scope.go:117] "RemoveContainer" containerID="78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f" Jul 06 22:47:34.877760 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:47:34.877737 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f\": container with ID starting with 78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f not found: ID does not exist" containerID="78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f" Jul 06 22:47:34.877858 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.877768 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f"} err="failed to get container status \"78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f\": rpc error: code = NotFound desc = could not find container \"78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f\": container with ID starting with 78b0f58e3459a8af20883fb542a5539bb11d76e92142f6ef2da6ef834ac5a65f not found: ID does not exist" Jul 06 22:47:34.877858 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.877788 2578 scope.go:117] "RemoveContainer" containerID="176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f" Jul 06 22:47:34.878020 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:47:34.878001 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f\": container with ID starting with 176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f not found: ID does not exist" containerID="176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f" Jul 06 22:47:34.878082 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.878030 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f"} err="failed to get container status \"176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f\": rpc error: code = NotFound desc = could not find container \"176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f\": container with ID starting with 176b29224c8fdf562b57362976e7dce5a5c64733916391c2d4561e5d6144f25f not found: ID does not exist" Jul 06 22:47:34.878818 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.878800 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr"] Jul 06 22:47:34.883546 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:34.883523 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-29698a60-openshift-default-65795557bc-kx4lr"] Jul 06 22:47:36.379499 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:36.379462 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" path="/var/lib/kubelet/pods/3ecdc025-de93-4732-b2a5-aba7bffb65e4/volumes" Jul 06 22:47:47.800365 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.800333 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f"] Jul 06 22:47:47.800827 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.800692 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerName="istio-proxy" Jul 06 22:47:47.800827 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.800703 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerName="istio-proxy" Jul 06 22:47:47.800827 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.800711 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerName="istio-proxy" Jul 06 22:47:47.800827 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.800717 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerName="istio-proxy" Jul 06 22:47:47.800827 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.800776 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerName="istio-proxy" Jul 06 22:47:47.803782 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.803758 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.805887 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.805865 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-09d58787-openshift-default-dockercfg-mtdvx\"" Jul 06 22:47:47.813569 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.813534 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f"] Jul 06 22:47:47.869458 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.869424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-istio-token\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.869458 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.869462 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpxf4\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-kube-api-access-tpxf4\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.869712 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.869498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-socket\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.869712 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.869535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-credential-socket\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.869712 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.869570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-envoy\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.869712 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.869598 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-data\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.869712 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.869630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-certs\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.869712 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.869665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c2a442-01cc-4873-8808-d6d942437e35-istiod-ca-cert\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.869712 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.869692 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c2a442-01cc-4873-8808-d6d942437e35-istio-podinfo\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.970655 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.970616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-data\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.970655 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.970656 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-certs\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.970882 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.970682 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c2a442-01cc-4873-8808-d6d942437e35-istiod-ca-cert\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.970882 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.970774 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c2a442-01cc-4873-8808-d6d942437e35-istio-podinfo\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.970882 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.970876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-istio-token\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.970990 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.970910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpxf4\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-kube-api-access-tpxf4\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.971026 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.970986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-socket\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.971026 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.971013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-credential-socket\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.971086 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.971040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-envoy\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.971086 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.971050 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-certs\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.971086 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.971070 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-data\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.971470 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.971446 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c2a442-01cc-4873-8808-d6d942437e35-istiod-ca-cert\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.971637 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.971483 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-socket\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.971765 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.971521 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-credential-socket\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.974193 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.974167 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c2a442-01cc-4873-8808-d6d942437e35-istio-podinfo\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.974399 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.974379 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-envoy\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.979857 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.979832 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-istio-token\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:47.980159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:47.980137 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpxf4\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-kube-api-access-tpxf4\") pod \"e2e-mt-09d58787-openshift-default-544574967-jqf4f\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:48.114816 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:48.114774 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:48.255952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:48.255923 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f"] Jul 06 22:47:48.257763 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:47:48.257735 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c2a442_01cc_4873_8808_d6d942437e35.slice/crio-a4a82d3f95f8cba516c87a9e73826e144a77adad7654eec89cd3838763b35fa4 WatchSource:0}: Error finding container a4a82d3f95f8cba516c87a9e73826e144a77adad7654eec89cd3838763b35fa4: Status 404 returned error can't find the container with id a4a82d3f95f8cba516c87a9e73826e144a77adad7654eec89cd3838763b35fa4 Jul 06 22:47:48.259974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:48.259944 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:47:48.260066 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:48.260004 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:47:48.260066 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:48.260029 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:47:48.933841 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:48.933808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" event={"ID":"a2c2a442-01cc-4873-8808-d6d942437e35","Type":"ContainerStarted","Data":"b2a2088def0bd2970100ab1a98473ff665732ed64a8ca26499d9fb3ec4c02889"} Jul 06 22:47:48.934302 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:48.933860 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" event={"ID":"a2c2a442-01cc-4873-8808-d6d942437e35","Type":"ContainerStarted","Data":"a4a82d3f95f8cba516c87a9e73826e144a77adad7654eec89cd3838763b35fa4"} Jul 06 22:47:48.956694 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:48.956637 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" podStartSLOduration=1.956621072 podStartE2EDuration="1.956621072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:47:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:47:48.955529072 +0000 UTC m=+1659.057256736" watchObservedRunningTime="2026-07-06 22:47:48.956621072 +0000 UTC m=+1659.058348803" Jul 06 22:47:49.115047 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:49.115008 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:49.120105 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:49.120079 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:49.939654 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:49.939620 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:47:49.940756 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:47:49.940734 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:48:05.703762 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:05.703160 2578 patch_prober.go:28] interesting pod/e2e-mt-09d58787-openshift-default-544574967-jqf4f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:45364->10.134.0.54:15021: read: connection reset by peer" start-of-body= Jul 06 22:48:05.703762 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:05.703218 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:45364->10.134.0.54:15021: read: connection reset by peer" Jul 06 22:48:06.033710 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:06.033606 2578 generic.go:356] "Generic (PLEG): container finished" podID="a2c2a442-01cc-4873-8808-d6d942437e35" containerID="b2a2088def0bd2970100ab1a98473ff665732ed64a8ca26499d9fb3ec4c02889" exitCode=0 Jul 06 22:48:06.033710 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:06.033658 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" event={"ID":"a2c2a442-01cc-4873-8808-d6d942437e35","Type":"ContainerDied","Data":"b2a2088def0bd2970100ab1a98473ff665732ed64a8ca26499d9fb3ec4c02889"} Jul 06 22:48:06.034021 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:06.033999 2578 scope.go:117] "RemoveContainer" containerID="b2a2088def0bd2970100ab1a98473ff665732ed64a8ca26499d9fb3ec4c02889" Jul 06 22:48:06.035039 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:06.035012 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:48:06.035094 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:06.035081 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:48:06.035148 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:06.035134 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:48:07.040985 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:07.040950 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" event={"ID":"a2c2a442-01cc-4873-8808-d6d942437e35","Type":"ContainerStarted","Data":"157c397ea23807061693c901e1201007252c901ed2a6b0ba6f29ce0403e518c1"} Jul 06 22:48:07.115304 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:07.115260 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:48:07.115620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:07.115598 2578 patch_prober.go:28] interesting pod/e2e-mt-09d58787-openshift-default-544574967-jqf4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 06 22:48:07.115672 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:07.115648 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 06 22:48:08.115700 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:08.115661 2578 patch_prober.go:28] interesting pod/e2e-mt-09d58787-openshift-default-544574967-jqf4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 06 22:48:08.116147 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:08.115720 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 06 22:48:09.115159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:09.115095 2578 patch_prober.go:28] interesting pod/e2e-mt-09d58787-openshift-default-544574967-jqf4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 06 22:48:09.115337 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:09.115180 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 06 22:48:10.115439 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:10.115399 2578 patch_prober.go:28] interesting pod/e2e-mt-09d58787-openshift-default-544574967-jqf4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 06 22:48:10.115845 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:10.115459 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 06 22:48:11.119451 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:11.119423 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:48:11.119826 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:11.119690 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:48:11.120513 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:48:11.120490 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:49:56.512137 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:49:56.512078 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f"] Jul 06 22:49:56.512706 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:49:56.512308 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" containerID="cri-o://157c397ea23807061693c901e1201007252c901ed2a6b0ba6f29ce0403e518c1" gracePeriod=30 Jul 06 22:50:01.653729 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.653695 2578 generic.go:356] "Generic (PLEG): container finished" podID="a2c2a442-01cc-4873-8808-d6d942437e35" containerID="157c397ea23807061693c901e1201007252c901ed2a6b0ba6f29ce0403e518c1" exitCode=0 Jul 06 22:50:01.654169 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.653767 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" event={"ID":"a2c2a442-01cc-4873-8808-d6d942437e35","Type":"ContainerDied","Data":"157c397ea23807061693c901e1201007252c901ed2a6b0ba6f29ce0403e518c1"} Jul 06 22:50:01.654169 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.653832 2578 scope.go:117] "RemoveContainer" containerID="b2a2088def0bd2970100ab1a98473ff665732ed64a8ca26499d9fb3ec4c02889" Jul 06 22:50:01.771054 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.770989 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:50:01.888952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.888901 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-istio-token\") pod \"a2c2a442-01cc-4873-8808-d6d942437e35\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " Jul 06 22:50:01.888952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.888956 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-credential-socket\") pod \"a2c2a442-01cc-4873-8808-d6d942437e35\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " Jul 06 22:50:01.889246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.888985 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c2a442-01cc-4873-8808-d6d942437e35-istio-podinfo\") pod \"a2c2a442-01cc-4873-8808-d6d942437e35\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " Jul 06 22:50:01.889246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889004 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c2a442-01cc-4873-8808-d6d942437e35-istiod-ca-cert\") pod \"a2c2a442-01cc-4873-8808-d6d942437e35\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " Jul 06 22:50:01.889246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889033 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-certs\") pod \"a2c2a442-01cc-4873-8808-d6d942437e35\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " Jul 06 22:50:01.889246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889059 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpxf4\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-kube-api-access-tpxf4\") pod \"a2c2a442-01cc-4873-8808-d6d942437e35\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " Jul 06 22:50:01.889246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889180 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-socket\") pod \"a2c2a442-01cc-4873-8808-d6d942437e35\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " Jul 06 22:50:01.889246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889210 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-envoy\") pod \"a2c2a442-01cc-4873-8808-d6d942437e35\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " Jul 06 22:50:01.889246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889238 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-data\") pod \"a2c2a442-01cc-4873-8808-d6d942437e35\" (UID: \"a2c2a442-01cc-4873-8808-d6d942437e35\") " Jul 06 22:50:01.889599 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889402 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2c2a442-01cc-4873-8808-d6d942437e35" (UID: "a2c2a442-01cc-4873-8808-d6d942437e35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:50:01.889599 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889421 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2c2a442-01cc-4873-8808-d6d942437e35" (UID: "a2c2a442-01cc-4873-8808-d6d942437e35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:50:01.889599 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889547 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2c2a442-01cc-4873-8808-d6d942437e35" (UID: "a2c2a442-01cc-4873-8808-d6d942437e35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:50:01.889599 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889572 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:50:01.889802 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889614 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:50:01.889802 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.889578 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2c2a442-01cc-4873-8808-d6d942437e35-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2c2a442-01cc-4873-8808-d6d942437e35" (UID: "a2c2a442-01cc-4873-8808-d6d942437e35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:50:01.890453 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.890429 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2c2a442-01cc-4873-8808-d6d942437e35" (UID: "a2c2a442-01cc-4873-8808-d6d942437e35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:50:01.891152 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.891134 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2c2a442-01cc-4873-8808-d6d942437e35" (UID: "a2c2a442-01cc-4873-8808-d6d942437e35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:50:01.891473 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.891449 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-kube-api-access-tpxf4" (OuterVolumeSpecName: "kube-api-access-tpxf4") pod "a2c2a442-01cc-4873-8808-d6d942437e35" (UID: "a2c2a442-01cc-4873-8808-d6d942437e35"). InnerVolumeSpecName "kube-api-access-tpxf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:50:01.891759 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.891741 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2c2a442-01cc-4873-8808-d6d942437e35" (UID: "a2c2a442-01cc-4873-8808-d6d942437e35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:50:01.891849 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.891827 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2c2a442-01cc-4873-8808-d6d942437e35-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2c2a442-01cc-4873-8808-d6d942437e35" (UID: "a2c2a442-01cc-4873-8808-d6d942437e35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:50:01.990103 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.990001 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpxf4\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-kube-api-access-tpxf4\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:50:01.990103 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.990041 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:50:01.990103 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.990055 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:50:01.990103 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.990067 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2c2a442-01cc-4873-8808-d6d942437e35-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:50:01.990103 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.990078 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2c2a442-01cc-4873-8808-d6d942437e35-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:50:01.990103 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.990089 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2c2a442-01cc-4873-8808-d6d942437e35-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:50:01.990103 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:01.990100 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2c2a442-01cc-4873-8808-d6d942437e35-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:50:02.659987 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:02.659904 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" Jul 06 22:50:02.659987 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:02.659954 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f" event={"ID":"a2c2a442-01cc-4873-8808-d6d942437e35","Type":"ContainerDied","Data":"a4a82d3f95f8cba516c87a9e73826e144a77adad7654eec89cd3838763b35fa4"} Jul 06 22:50:02.660479 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:02.660019 2578 scope.go:117] "RemoveContainer" containerID="157c397ea23807061693c901e1201007252c901ed2a6b0ba6f29ce0403e518c1" Jul 06 22:50:02.680202 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:02.680176 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f"] Jul 06 22:50:02.684823 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:02.684797 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-09d58787-openshift-default-544574967-jqf4f"] Jul 06 22:50:04.378819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:04.378773 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" path="/var/lib/kubelet/pods/a2c2a442-01cc-4873-8808-d6d942437e35/volumes" Jul 06 22:50:52.197186 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.197145 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r"] Jul 06 22:50:52.197668 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.197537 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" Jul 06 22:50:52.197668 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.197549 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" Jul 06 22:50:52.197668 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.197564 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" Jul 06 22:50:52.197668 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.197569 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" Jul 06 22:50:52.197668 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.197632 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" Jul 06 22:50:52.197668 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.197641 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ecdc025-de93-4732-b2a5-aba7bffb65e4" containerName="istio-proxy" Jul 06 22:50:52.200806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.200781 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.202977 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.202953 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-eb8d59-openshift-default-dockercfg-qv4nk\"" Jul 06 22:50:52.212019 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.211982 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-data\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.212285 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.212260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f074e41-8bc3-4de1-84be-fe8418659c34-istiod-ca-cert\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.212402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.212296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-socket\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.212402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.212331 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-credential-socket\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.212402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.212361 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-certs\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.212402 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.212382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-podinfo\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.212623 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.212403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rknvp\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-kube-api-access-rknvp\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.212623 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.212441 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-token\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.212623 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.212470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-envoy\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.213511 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.213490 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r"] Jul 06 22:50:52.313279 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313243 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-podinfo\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.313279 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rknvp\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-kube-api-access-rknvp\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.313535 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313314 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-token\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.313535 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-envoy\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.313535 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313374 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-data\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.313535 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f074e41-8bc3-4de1-84be-fe8418659c34-istiod-ca-cert\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.313750 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-socket\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.313750 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313582 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-credential-socket\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.313750 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-certs\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.314011 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.313980 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-data\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.314097 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.314014 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-certs\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.314182 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.314090 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-socket\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.314182 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.314144 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-credential-socket\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.314398 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.314373 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f074e41-8bc3-4de1-84be-fe8418659c34-istiod-ca-cert\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.316124 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.316097 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-podinfo\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.316201 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.316095 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-envoy\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.323979 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.323952 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-token\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.324145 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.324012 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rknvp\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-kube-api-access-rknvp\") pod \"e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.513786 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.513694 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:52.664912 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.664883 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r"] Jul 06 22:50:52.667328 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:50:52.667263 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f074e41_8bc3_4de1_84be_fe8418659c34.slice/crio-b6c958b3ead001a772acd83692582824fe1e9a4cf795c1f665ed5559eafc4786 WatchSource:0}: Error finding container b6c958b3ead001a772acd83692582824fe1e9a4cf795c1f665ed5559eafc4786: Status 404 returned error can't find the container with id b6c958b3ead001a772acd83692582824fe1e9a4cf795c1f665ed5559eafc4786 Jul 06 22:50:52.670505 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.670465 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:50:52.670624 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.670566 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:50:52.670687 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.670619 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:50:52.929845 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.929803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" event={"ID":"7f074e41-8bc3-4de1-84be-fe8418659c34","Type":"ContainerStarted","Data":"03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9"} Jul 06 22:50:52.930058 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.929864 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" event={"ID":"7f074e41-8bc3-4de1-84be-fe8418659c34","Type":"ContainerStarted","Data":"b6c958b3ead001a772acd83692582824fe1e9a4cf795c1f665ed5559eafc4786"} Jul 06 22:50:52.953166 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:52.953102 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" podStartSLOduration=0.953085629 podStartE2EDuration="953.085629ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:50:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:50:52.950956161 +0000 UTC m=+1843.052683824" watchObservedRunningTime="2026-07-06 22:50:52.953085629 +0000 UTC m=+1843.054813344" Jul 06 22:50:53.514326 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:53.514283 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:53.519412 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:53.519384 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:53.935535 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:53.935500 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:53.936508 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:53.936483 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:50:59.977733 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:59.977692 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq"] Jul 06 22:50:59.978676 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:59.978653 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c2a442-01cc-4873-8808-d6d942437e35" containerName="istio-proxy" Jul 06 22:50:59.983183 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:59.983156 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:50:59.985780 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:59.985754 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e401d1-openshift-default-dockercfg-kl7gh\"" Jul 06 22:50:59.989936 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:50:59.989912 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq"] Jul 06 22:51:00.084005 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.083965 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-podinfo\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.084005 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.084008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9fs8\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-kube-api-access-f9fs8\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.084242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.084069 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-data\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.084242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.084086 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-certs\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.084242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.084122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-socket\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.084242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.084167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-envoy\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.084374 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.084243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-token\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.084374 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.084276 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-credential-socket\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.084374 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.084296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istiod-ca-cert\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185139 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-token\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185325 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185174 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-credential-socket\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185325 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185196 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istiod-ca-cert\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185325 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-podinfo\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185325 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185254 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9fs8\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-kube-api-access-f9fs8\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185325 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-data\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185593 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-certs\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185593 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185360 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-socket\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185593 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185393 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-envoy\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185719 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185660 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-credential-socket\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.185865 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185836 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-data\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.186004 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185885 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-certs\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.186004 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.185916 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-socket\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.186220 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.186199 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istiod-ca-cert\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.187720 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.187700 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-envoy\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.187859 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.187840 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-podinfo\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.193749 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.193721 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9fs8\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-kube-api-access-f9fs8\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.194139 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.194098 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-token\") pod \"e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.295715 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.295618 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:00.446975 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.446947 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq"] Jul 06 22:51:00.448796 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:51:00.448767 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be79e6b_adc7_4dd1_813b_5987a74bafeb.slice/crio-80020dfe203b780b2088a64c2737e7409a00f058c351b1c31b89cb05c000e474 WatchSource:0}: Error finding container 80020dfe203b780b2088a64c2737e7409a00f058c351b1c31b89cb05c000e474: Status 404 returned error can't find the container with id 80020dfe203b780b2088a64c2737e7409a00f058c351b1c31b89cb05c000e474 Jul 06 22:51:00.451048 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.451019 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:51:00.451159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.451082 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:51:00.451159 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.451134 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:51:00.978478 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.978432 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" event={"ID":"0be79e6b-adc7-4dd1-813b-5987a74bafeb","Type":"ContainerStarted","Data":"b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c"} Jul 06 22:51:00.978947 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.978493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" event={"ID":"0be79e6b-adc7-4dd1-813b-5987a74bafeb","Type":"ContainerStarted","Data":"80020dfe203b780b2088a64c2737e7409a00f058c351b1c31b89cb05c000e474"} Jul 06 22:51:00.999713 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:00.999661 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" podStartSLOduration=1.999639983 podStartE2EDuration="1.999639983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:50:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:51:00.998870621 +0000 UTC m=+1851.100598284" watchObservedRunningTime="2026-07-06 22:51:00.999639983 +0000 UTC m=+1851.101367646" Jul 06 22:51:01.296754 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:01.296661 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:01.301819 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:01.301794 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:01.983647 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:01.983617 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:01.984775 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:01.984754 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:09.977401 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:09.977350 2578 patch_prober.go:28] interesting pod/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 22:51:09.978024 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:09.977987 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 22:51:11.036388 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:11.036299 2578 generic.go:356] "Generic (PLEG): container finished" podID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerID="03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9" exitCode=0 Jul 06 22:51:11.036388 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:11.036374 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" event={"ID":"7f074e41-8bc3-4de1-84be-fe8418659c34","Type":"ContainerDied","Data":"03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9"} Jul 06 22:51:11.036833 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:11.036778 2578 scope.go:117] "RemoveContainer" containerID="03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9" Jul 06 22:51:11.037874 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:11.037849 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:51:11.037941 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:11.037912 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:51:11.037977 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:11.037943 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:51:12.043008 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:12.042970 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" event={"ID":"7f074e41-8bc3-4de1-84be-fe8418659c34","Type":"ContainerStarted","Data":"7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c"} Jul 06 22:51:12.514374 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:12.514336 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:51:12.514696 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:12.514669 2578 patch_prober.go:28] interesting pod/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 06 22:51:12.514767 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:12.514730 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 06 22:51:13.514351 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:13.514290 2578 patch_prober.go:28] interesting pod/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 06 22:51:13.514885 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:13.514364 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 06 22:51:14.514653 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:14.514609 2578 patch_prober.go:28] interesting pod/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 06 22:51:14.515095 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:14.514679 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 06 22:51:15.514199 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:15.514150 2578 patch_prober.go:28] interesting pod/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 06 22:51:15.514376 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:15.514226 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 06 22:51:16.518017 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:16.517985 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:51:16.518507 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:16.518495 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:51:16.519279 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:16.519253 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:51:17.264475 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:17.263764 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:41226->10.134.0.56:15021: read: connection reset by peer" start-of-body= Jul 06 22:51:17.264475 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:17.263854 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": read tcp 10.134.0.2:41226->10.134.0.56:15021: read: connection reset by peer" Jul 06 22:51:18.078423 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:18.078387 2578 generic.go:356] "Generic (PLEG): container finished" podID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerID="b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c" exitCode=0 Jul 06 22:51:18.078865 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:18.078461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" event={"ID":"0be79e6b-adc7-4dd1-813b-5987a74bafeb","Type":"ContainerDied","Data":"b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c"} Jul 06 22:51:18.079015 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:18.078983 2578 scope.go:117] "RemoveContainer" containerID="b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c" Jul 06 22:51:18.080024 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:18.079996 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:51:18.080161 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:18.080059 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:51:18.080161 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:18.080093 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:51:19.091349 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:19.091310 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" event={"ID":"0be79e6b-adc7-4dd1-813b-5987a74bafeb","Type":"ContainerStarted","Data":"e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91"} Jul 06 22:51:19.296540 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:19.296492 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:19.296775 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:19.296751 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 06 22:51:19.296826 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:19.296804 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 06 22:51:20.296364 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:20.296327 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 06 22:51:20.296778 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:20.296388 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 06 22:51:21.296224 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:21.296184 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 06 22:51:21.296419 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:21.296244 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 06 22:51:22.296486 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:22.296454 2578 patch_prober.go:28] interesting pod/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 06 22:51:22.296968 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:22.296944 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 06 22:51:23.300306 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:23.300272 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:23.300792 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:23.300729 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:51:23.301472 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:51:23.301450 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:53:05.666501 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:05.666415 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r"] Jul 06 22:53:05.667040 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:05.666719 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" containerID="cri-o://7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c" gracePeriod=30 Jul 06 22:53:10.919799 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.919767 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.994575 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-data\") pod \"7f074e41-8bc3-4de1-84be-fe8418659c34\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.994645 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rknvp\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-kube-api-access-rknvp\") pod \"7f074e41-8bc3-4de1-84be-fe8418659c34\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.994681 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-envoy\") pod \"7f074e41-8bc3-4de1-84be-fe8418659c34\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.994733 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-token\") pod \"7f074e41-8bc3-4de1-84be-fe8418659c34\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.994785 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-podinfo\") pod \"7f074e41-8bc3-4de1-84be-fe8418659c34\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.994853 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-socket\") pod \"7f074e41-8bc3-4de1-84be-fe8418659c34\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.994887 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f074e41-8bc3-4de1-84be-fe8418659c34-istiod-ca-cert\") pod \"7f074e41-8bc3-4de1-84be-fe8418659c34\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.994917 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-credential-socket\") pod \"7f074e41-8bc3-4de1-84be-fe8418659c34\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.994952 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-certs\") pod \"7f074e41-8bc3-4de1-84be-fe8418659c34\" (UID: \"7f074e41-8bc3-4de1-84be-fe8418659c34\") " Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.995610 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f074e41-8bc3-4de1-84be-fe8418659c34" (UID: "7f074e41-8bc3-4de1-84be-fe8418659c34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:10.999753 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.996453 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f074e41-8bc3-4de1-84be-fe8418659c34" (UID: "7f074e41-8bc3-4de1-84be-fe8418659c34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:11.000328 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.999855 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f074e41-8bc3-4de1-84be-fe8418659c34" (UID: "7f074e41-8bc3-4de1-84be-fe8418659c34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:11.000328 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.999866 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f074e41-8bc3-4de1-84be-fe8418659c34" (UID: "7f074e41-8bc3-4de1-84be-fe8418659c34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:11.000328 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:10.999918 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f074e41-8bc3-4de1-84be-fe8418659c34-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f074e41-8bc3-4de1-84be-fe8418659c34" (UID: "7f074e41-8bc3-4de1-84be-fe8418659c34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:53:11.000328 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.000057 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-kube-api-access-rknvp" (OuterVolumeSpecName: "kube-api-access-rknvp") pod "7f074e41-8bc3-4de1-84be-fe8418659c34" (UID: "7f074e41-8bc3-4de1-84be-fe8418659c34"). InnerVolumeSpecName "kube-api-access-rknvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:11.001510 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.001483 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f074e41-8bc3-4de1-84be-fe8418659c34" (UID: "7f074e41-8bc3-4de1-84be-fe8418659c34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:53:11.001809 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.001787 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f074e41-8bc3-4de1-84be-fe8418659c34" (UID: "7f074e41-8bc3-4de1-84be-fe8418659c34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:11.002227 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.002208 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f074e41-8bc3-4de1-84be-fe8418659c34" (UID: "7f074e41-8bc3-4de1-84be-fe8418659c34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:11.096527 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.096487 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:11.096527 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.096521 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f074e41-8bc3-4de1-84be-fe8418659c34-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:11.096527 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.096530 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:11.096527 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.096539 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:11.096806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.096548 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:11.096806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.096557 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rknvp\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-kube-api-access-rknvp\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:11.096806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.096566 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:11.096806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.096574 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:11.096806 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.096581 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f074e41-8bc3-4de1-84be-fe8418659c34-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:11.706905 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.706872 2578 generic.go:356] "Generic (PLEG): container finished" podID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerID="7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c" exitCode=0 Jul 06 22:53:11.707147 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.706927 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" event={"ID":"7f074e41-8bc3-4de1-84be-fe8418659c34","Type":"ContainerDied","Data":"7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c"} Jul 06 22:53:11.707147 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.706938 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" Jul 06 22:53:11.707147 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.706982 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r" event={"ID":"7f074e41-8bc3-4de1-84be-fe8418659c34","Type":"ContainerDied","Data":"b6c958b3ead001a772acd83692582824fe1e9a4cf795c1f665ed5559eafc4786"} Jul 06 22:53:11.707147 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.707012 2578 scope.go:117] "RemoveContainer" containerID="7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c" Jul 06 22:53:11.716946 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.716927 2578 scope.go:117] "RemoveContainer" containerID="03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9" Jul 06 22:53:11.725628 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.725610 2578 scope.go:117] "RemoveContainer" containerID="7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c" Jul 06 22:53:11.725893 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:53:11.725877 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c\": container with ID starting with 7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c not found: ID does not exist" containerID="7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c" Jul 06 22:53:11.725947 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.725901 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c"} err="failed to get container status \"7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c\": rpc error: code = NotFound desc = could not find container \"7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c\": container with ID starting with 7b8124d7565b08927f9c881a1dfc09ecb753ab56b7faa80211e5e663e25e334c not found: ID does not exist" Jul 06 22:53:11.725947 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.725917 2578 scope.go:117] "RemoveContainer" containerID="03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9" Jul 06 22:53:11.726209 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:53:11.726188 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9\": container with ID starting with 03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9 not found: ID does not exist" containerID="03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9" Jul 06 22:53:11.726263 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.726215 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9"} err="failed to get container status \"03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9\": rpc error: code = NotFound desc = could not find container \"03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9\": container with ID starting with 03de653e3226dac3ccac2452193d02bc2a82510b0e93cf61dd8f8ab86c6517c9 not found: ID does not exist" Jul 06 22:53:11.730094 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.730071 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r"] Jul 06 22:53:11.734198 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:11.734176 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-eb8d59-openshift-default-6cb9d55dd7-4l97r"] Jul 06 22:53:12.379667 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:12.379633 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" path="/var/lib/kubelet/pods/7f074e41-8bc3-4de1-84be-fe8418659c34/volumes" Jul 06 22:53:13.089035 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.089002 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq"] Jul 06 22:53:13.089274 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.089246 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" containerID="cri-o://e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91" gracePeriod=30 Jul 06 22:53:13.958888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.958855 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9"] Jul 06 22:53:13.959447 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.959429 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" Jul 06 22:53:13.959533 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.959451 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" Jul 06 22:53:13.959584 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.959574 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" Jul 06 22:53:13.959637 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.959587 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" Jul 06 22:53:13.962562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.962543 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:13.964783 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.964763 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8f36f0-openshift-default-dockercfg-9dmtb\"" Jul 06 22:53:13.974492 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:13.974199 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9"] Jul 06 22:53:14.016811 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.016767 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-podinfo\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.016811 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.016820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f102fb-f391-47cc-8a2b-b60c94478a18-istiod-ca-cert\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.017033 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.016891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-credential-socket\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.017033 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.016953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-envoy\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.017033 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.016989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8qsw\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-kube-api-access-r8qsw\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.017033 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.017015 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-socket\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.017224 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.017035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-certs\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.017224 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.017053 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-data\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.017224 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.017082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-token\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.117557 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.117517 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-socket\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.117746 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.117641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-certs\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.117746 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.117676 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-data\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.117746 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.117712 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-token\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.117746 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.117741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-podinfo\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.117974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.117777 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f102fb-f391-47cc-8a2b-b60c94478a18-istiod-ca-cert\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.117974 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.117897 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-socket\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.118078 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.117998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-credential-socket\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.118078 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.118015 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-certs\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.118196 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.118084 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-envoy\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.118196 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.118139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8qsw\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-kube-api-access-r8qsw\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.118298 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.118239 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-credential-socket\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.118566 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.118542 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f102fb-f391-47cc-8a2b-b60c94478a18-istiod-ca-cert\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.118697 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.118644 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-data\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.120211 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.120189 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-envoy\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.120317 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.120300 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-podinfo\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.126672 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.126642 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-token\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.126784 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.126764 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8qsw\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-kube-api-access-r8qsw\") pod \"e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.275761 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.275673 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:14.421281 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.421251 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9"] Jul 06 22:53:14.424566 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:53:14.424539 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f102fb_f391_47cc_8a2b_b60c94478a18.slice/crio-7b442bf4de31d116221631aa681d1f017d3a65b1e1b9485e8c0a4e52e73272dc WatchSource:0}: Error finding container 7b442bf4de31d116221631aa681d1f017d3a65b1e1b9485e8c0a4e52e73272dc: Status 404 returned error can't find the container with id 7b442bf4de31d116221631aa681d1f017d3a65b1e1b9485e8c0a4e52e73272dc Jul 06 22:53:14.426798 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.426754 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:14.426889 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.426849 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:14.426928 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.426896 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:14.725541 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.725498 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" event={"ID":"e2f102fb-f391-47cc-8a2b-b60c94478a18","Type":"ContainerStarted","Data":"9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe"} Jul 06 22:53:14.725727 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.725569 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" event={"ID":"e2f102fb-f391-47cc-8a2b-b60c94478a18","Type":"ContainerStarted","Data":"7b442bf4de31d116221631aa681d1f017d3a65b1e1b9485e8c0a4e52e73272dc"} Jul 06 22:53:14.745550 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:14.745500 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" podStartSLOduration=1.745484044 podStartE2EDuration="1.745484044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:53:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:14.743862433 +0000 UTC m=+1984.845590095" watchObservedRunningTime="2026-07-06 22:53:14.745484044 +0000 UTC m=+1984.847211706" Jul 06 22:53:15.276533 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:15.276489 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:15.281465 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:15.281440 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:15.730787 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:15.730757 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:15.731638 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:15.731619 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:18.350042 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.350010 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:53:18.461859 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.461776 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-credential-socket\") pod \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " Jul 06 22:53:18.461859 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.461816 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-data\") pod \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " Jul 06 22:53:18.461859 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.461859 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-envoy\") pod \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " Jul 06 22:53:18.462172 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.461887 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-podinfo\") pod \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " Jul 06 22:53:18.462172 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.461914 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9fs8\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-kube-api-access-f9fs8\") pod \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " Jul 06 22:53:18.462172 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462081 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istiod-ca-cert\") pod \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " Jul 06 22:53:18.462172 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462102 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0be79e6b-adc7-4dd1-813b-5987a74bafeb" (UID: "0be79e6b-adc7-4dd1-813b-5987a74bafeb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:18.462172 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462156 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-token\") pod \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " Jul 06 22:53:18.462467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462197 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-socket\") pod \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " Jul 06 22:53:18.462467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462230 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-certs\") pod \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\" (UID: \"0be79e6b-adc7-4dd1-813b-5987a74bafeb\") " Jul 06 22:53:18.462570 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462495 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0be79e6b-adc7-4dd1-813b-5987a74bafeb" (UID: "0be79e6b-adc7-4dd1-813b-5987a74bafeb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:18.463242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462670 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:18.463242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462698 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:18.463242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462767 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-data" (OuterVolumeSpecName: "istio-data") pod "0be79e6b-adc7-4dd1-813b-5987a74bafeb" (UID: "0be79e6b-adc7-4dd1-813b-5987a74bafeb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:18.463242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.462971 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0be79e6b-adc7-4dd1-813b-5987a74bafeb" (UID: "0be79e6b-adc7-4dd1-813b-5987a74bafeb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:18.463242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.463059 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0be79e6b-adc7-4dd1-813b-5987a74bafeb" (UID: "0be79e6b-adc7-4dd1-813b-5987a74bafeb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:53:18.464387 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.464362 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0be79e6b-adc7-4dd1-813b-5987a74bafeb" (UID: "0be79e6b-adc7-4dd1-813b-5987a74bafeb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:18.464387 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.464371 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-token" (OuterVolumeSpecName: "istio-token") pod "0be79e6b-adc7-4dd1-813b-5987a74bafeb" (UID: "0be79e6b-adc7-4dd1-813b-5987a74bafeb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:18.464553 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.464493 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-kube-api-access-f9fs8" (OuterVolumeSpecName: "kube-api-access-f9fs8") pod "0be79e6b-adc7-4dd1-813b-5987a74bafeb" (UID: "0be79e6b-adc7-4dd1-813b-5987a74bafeb"). InnerVolumeSpecName "kube-api-access-f9fs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:18.464553 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.464517 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0be79e6b-adc7-4dd1-813b-5987a74bafeb" (UID: "0be79e6b-adc7-4dd1-813b-5987a74bafeb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:53:18.563560 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.563513 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:18.563560 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.563554 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:18.563560 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.563563 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:18.563807 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.563572 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:18.563807 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.563581 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9fs8\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-kube-api-access-f9fs8\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:18.563807 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.563589 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:18.563807 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.563597 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be79e6b-adc7-4dd1-813b-5987a74bafeb-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:18.764183 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.764086 2578 generic.go:356] "Generic (PLEG): container finished" podID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerID="e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91" exitCode=0 Jul 06 22:53:18.764183 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.764148 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" Jul 06 22:53:18.764183 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.764161 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" event={"ID":"0be79e6b-adc7-4dd1-813b-5987a74bafeb","Type":"ContainerDied","Data":"e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91"} Jul 06 22:53:18.764441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.764210 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq" event={"ID":"0be79e6b-adc7-4dd1-813b-5987a74bafeb","Type":"ContainerDied","Data":"80020dfe203b780b2088a64c2737e7409a00f058c351b1c31b89cb05c000e474"} Jul 06 22:53:18.764441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.764239 2578 scope.go:117] "RemoveContainer" containerID="e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91" Jul 06 22:53:18.773740 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.773716 2578 scope.go:117] "RemoveContainer" containerID="b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c" Jul 06 22:53:18.781945 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.781925 2578 scope.go:117] "RemoveContainer" containerID="e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91" Jul 06 22:53:18.782234 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:53:18.782216 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91\": container with ID starting with e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91 not found: ID does not exist" containerID="e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91" Jul 06 22:53:18.782315 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.782241 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91"} err="failed to get container status \"e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91\": rpc error: code = NotFound desc = could not find container \"e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91\": container with ID starting with e24b6583daede78978499c8c14004a1f56a90710927fa640602b67aa6fa2db91 not found: ID does not exist" Jul 06 22:53:18.782315 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.782259 2578 scope.go:117] "RemoveContainer" containerID="b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c" Jul 06 22:53:18.782485 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:53:18.782464 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c\": container with ID starting with b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c not found: ID does not exist" containerID="b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c" Jul 06 22:53:18.782523 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.782496 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c"} err="failed to get container status \"b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c\": rpc error: code = NotFound desc = could not find container \"b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c\": container with ID starting with b4e6c19109e00cae2244033b23aa01cdb2ef71d793b73614c2d1e6e438bf3d1c not found: ID does not exist" Jul 06 22:53:18.787195 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.787174 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq"] Jul 06 22:53:18.793557 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:18.793537 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e401d1-openshift-default-64f6d967b5-72slq"] Jul 06 22:53:20.379941 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:20.379899 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" path="/var/lib/kubelet/pods/0be79e6b-adc7-4dd1-813b-5987a74bafeb/volumes" Jul 06 22:53:21.936853 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.936820 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv"] Jul 06 22:53:21.937298 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.937278 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" Jul 06 22:53:21.937401 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.937301 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" Jul 06 22:53:21.937401 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.937318 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" Jul 06 22:53:21.937401 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.937326 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f074e41-8bc3-4de1-84be-fe8418659c34" containerName="istio-proxy" Jul 06 22:53:21.937621 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.937441 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" Jul 06 22:53:21.940966 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.940948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:21.943362 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.943336 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-164a1c-openshift-default-dockercfg-nzn2r\"" Jul 06 22:53:21.961258 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.961230 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv"] Jul 06 22:53:21.993636 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.993597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-podinfo\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:21.993805 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.993644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-credential-socket\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:21.993805 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.993731 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-data\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:21.993805 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.993774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkbvw\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-kube-api-access-tkbvw\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:21.993805 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.993795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-envoy\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:21.993945 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.993827 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-token\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:21.993945 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.993882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-socket\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:21.993945 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.993903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-certs\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:21.993945 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:21.993928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istiod-ca-cert\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.094936 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.094895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-token\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.094936 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.094941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-socket\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095214 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.094960 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-certs\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095214 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istiod-ca-cert\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095214 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095160 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-podinfo\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095214 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095202 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-credential-socket\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095426 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-data\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095426 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095333 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-socket\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095426 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095380 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-certs\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095581 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095523 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-credential-socket\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095581 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkbvw\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-kube-api-access-tkbvw\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095581 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-envoy\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095734 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095678 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-data\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.095865 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.095837 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istiod-ca-cert\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.097512 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.097495 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-envoy\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.097734 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.097716 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-podinfo\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.104158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.104103 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-token\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.104265 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.104223 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkbvw\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-kube-api-access-tkbvw\") pod \"e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.260852 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.260752 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:22.404106 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.404077 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv"] Jul 06 22:53:22.405103 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:53:22.405067 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc91a7d93_aeb4_4ac9_aaa6_48b9f1035d1b.slice/crio-bc2dd925c6efd41d1677a31e105324025150c263bcd470789487167eda532651 WatchSource:0}: Error finding container bc2dd925c6efd41d1677a31e105324025150c263bcd470789487167eda532651: Status 404 returned error can't find the container with id bc2dd925c6efd41d1677a31e105324025150c263bcd470789487167eda532651 Jul 06 22:53:22.407225 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.407193 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:22.407319 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.407274 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:22.407390 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.407317 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:22.792248 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.792211 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" event={"ID":"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b","Type":"ContainerStarted","Data":"70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24"} Jul 06 22:53:22.792429 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.792263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" event={"ID":"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b","Type":"ContainerStarted","Data":"bc2dd925c6efd41d1677a31e105324025150c263bcd470789487167eda532651"} Jul 06 22:53:22.814914 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:22.814861 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" podStartSLOduration=1.814844984 podStartE2EDuration="1.814844984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:53:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:22.812010946 +0000 UTC m=+1992.913738635" watchObservedRunningTime="2026-07-06 22:53:22.814844984 +0000 UTC m=+1992.916572646" Jul 06 22:53:23.261381 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:23.261344 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:23.266709 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:23.266680 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:23.797837 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:23.797805 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:23.798875 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:23.798857 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:31.773316 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:31.773272 2578 patch_prober.go:28] interesting pod/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 22:53:31.773721 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:31.773340 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 22:53:32.852171 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:32.852140 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" event={"ID":"e2f102fb-f391-47cc-8a2b-b60c94478a18","Type":"ContainerDied","Data":"9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe"} Jul 06 22:53:32.852651 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:32.852139 2578 generic.go:356] "Generic (PLEG): container finished" podID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerID="9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe" exitCode=0 Jul 06 22:53:32.852651 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:32.852544 2578 scope.go:117] "RemoveContainer" containerID="9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe" Jul 06 22:53:32.853493 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:32.853466 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:32.853593 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:32.853531 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:32.853593 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:32.853573 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:33.859320 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:33.859281 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" event={"ID":"e2f102fb-f391-47cc-8a2b-b60c94478a18","Type":"ContainerStarted","Data":"5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33"} Jul 06 22:53:34.276275 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:34.276155 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:34.276665 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:34.276628 2578 patch_prober.go:28] interesting pod/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 06 22:53:34.276786 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:34.276695 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 06 22:53:35.276835 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:35.276788 2578 patch_prober.go:28] interesting pod/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 06 22:53:35.277381 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:35.276865 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 06 22:53:36.277146 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:36.277085 2578 patch_prober.go:28] interesting pod/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 06 22:53:36.277673 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:36.277179 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 06 22:53:37.277161 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:37.277099 2578 patch_prober.go:28] interesting pod/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 06 22:53:37.277859 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:37.277827 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 06 22:53:38.277549 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:38.277010 2578 patch_prober.go:28] interesting pod/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 06 22:53:38.277549 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:38.277092 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 06 22:53:39.280171 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:39.280132 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:39.280555 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:39.280537 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:39.289168 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:39.281448 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:39.839278 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:39.839243 2578 patch_prober.go:28] interesting pod/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 22:53:39.839464 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:39.839316 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 22:53:40.910177 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:40.910080 2578 generic.go:356] "Generic (PLEG): container finished" podID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerID="70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24" exitCode=0 Jul 06 22:53:40.910564 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:40.910164 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" event={"ID":"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b","Type":"ContainerDied","Data":"70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24"} Jul 06 22:53:40.910707 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:40.910690 2578 scope.go:117] "RemoveContainer" containerID="70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24" Jul 06 22:53:40.911638 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:40.911613 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:40.911722 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:40.911676 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:40.911722 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:40.911706 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:41.917794 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:41.917758 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" event={"ID":"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b","Type":"ContainerStarted","Data":"543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b"} Jul 06 22:53:42.261183 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:42.261070 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:42.261418 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:42.261395 2578 patch_prober.go:28] interesting pod/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 06 22:53:42.261467 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:42.261448 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 06 22:53:43.262242 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:43.262203 2578 patch_prober.go:28] interesting pod/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 06 22:53:43.263246 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:43.262812 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 06 22:53:44.102523 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:44.102481 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9"] Jul 06 22:53:44.103100 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:44.103065 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" containerID="cri-o://5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33" gracePeriod=30 Jul 06 22:53:44.261173 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:44.261137 2578 patch_prober.go:28] interesting pod/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 06 22:53:44.261354 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:44.261207 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 06 22:53:45.262062 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:45.262022 2578 patch_prober.go:28] interesting pod/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 06 22:53:45.262606 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:45.262088 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 06 22:53:46.266284 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:46.266252 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:46.266713 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:46.266582 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:46.267354 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:46.267333 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:49.368855 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.368787 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:49.451826 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.451789 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-credential-socket\") pod \"e2f102fb-f391-47cc-8a2b-b60c94478a18\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " Jul 06 22:53:49.451994 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.451859 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f102fb-f391-47cc-8a2b-b60c94478a18-istiod-ca-cert\") pod \"e2f102fb-f391-47cc-8a2b-b60c94478a18\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " Jul 06 22:53:49.451994 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.451898 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-envoy\") pod \"e2f102fb-f391-47cc-8a2b-b60c94478a18\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " Jul 06 22:53:49.451994 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.451924 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8qsw\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-kube-api-access-r8qsw\") pod \"e2f102fb-f391-47cc-8a2b-b60c94478a18\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " Jul 06 22:53:49.451994 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.451960 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-certs\") pod \"e2f102fb-f391-47cc-8a2b-b60c94478a18\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " Jul 06 22:53:49.451994 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.451982 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-token\") pod \"e2f102fb-f391-47cc-8a2b-b60c94478a18\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " Jul 06 22:53:49.452312 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452028 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-podinfo\") pod \"e2f102fb-f391-47cc-8a2b-b60c94478a18\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " Jul 06 22:53:49.452312 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452082 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-data\") pod \"e2f102fb-f391-47cc-8a2b-b60c94478a18\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " Jul 06 22:53:49.452312 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452148 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e2f102fb-f391-47cc-8a2b-b60c94478a18" (UID: "e2f102fb-f391-47cc-8a2b-b60c94478a18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:49.452312 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452164 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-socket\") pod \"e2f102fb-f391-47cc-8a2b-b60c94478a18\" (UID: \"e2f102fb-f391-47cc-8a2b-b60c94478a18\") " Jul 06 22:53:49.452523 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452371 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f102fb-f391-47cc-8a2b-b60c94478a18-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e2f102fb-f391-47cc-8a2b-b60c94478a18" (UID: "e2f102fb-f391-47cc-8a2b-b60c94478a18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:53:49.452523 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452375 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e2f102fb-f391-47cc-8a2b-b60c94478a18" (UID: "e2f102fb-f391-47cc-8a2b-b60c94478a18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:49.452630 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452537 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e2f102fb-f391-47cc-8a2b-b60c94478a18" (UID: "e2f102fb-f391-47cc-8a2b-b60c94478a18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:49.452630 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452581 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:49.452630 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452602 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f102fb-f391-47cc-8a2b-b60c94478a18-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:49.452630 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.452615 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:49.453233 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.453211 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-data" (OuterVolumeSpecName: "istio-data") pod "e2f102fb-f391-47cc-8a2b-b60c94478a18" (UID: "e2f102fb-f391-47cc-8a2b-b60c94478a18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:49.454411 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.454387 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-kube-api-access-r8qsw" (OuterVolumeSpecName: "kube-api-access-r8qsw") pod "e2f102fb-f391-47cc-8a2b-b60c94478a18" (UID: "e2f102fb-f391-47cc-8a2b-b60c94478a18"). InnerVolumeSpecName "kube-api-access-r8qsw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:49.454521 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.454415 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e2f102fb-f391-47cc-8a2b-b60c94478a18" (UID: "e2f102fb-f391-47cc-8a2b-b60c94478a18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:53:49.454808 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.454785 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e2f102fb-f391-47cc-8a2b-b60c94478a18" (UID: "e2f102fb-f391-47cc-8a2b-b60c94478a18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:49.454808 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.454798 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2f102fb-f391-47cc-8a2b-b60c94478a18" (UID: "e2f102fb-f391-47cc-8a2b-b60c94478a18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:49.553824 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.553789 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:49.553824 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.553818 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:49.553824 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.553828 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:49.554056 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.553837 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8qsw\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-kube-api-access-r8qsw\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:49.554056 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.553847 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:49.554056 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.553855 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f102fb-f391-47cc-8a2b-b60c94478a18-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:49.962400 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.962358 2578 generic.go:356] "Generic (PLEG): container finished" podID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerID="5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33" exitCode=0 Jul 06 22:53:49.962579 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.962414 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" Jul 06 22:53:49.962579 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.962442 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" event={"ID":"e2f102fb-f391-47cc-8a2b-b60c94478a18","Type":"ContainerDied","Data":"5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33"} Jul 06 22:53:49.962579 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.962501 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9" event={"ID":"e2f102fb-f391-47cc-8a2b-b60c94478a18","Type":"ContainerDied","Data":"7b442bf4de31d116221631aa681d1f017d3a65b1e1b9485e8c0a4e52e73272dc"} Jul 06 22:53:49.962579 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.962524 2578 scope.go:117] "RemoveContainer" containerID="5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33" Jul 06 22:53:49.972580 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.972558 2578 scope.go:117] "RemoveContainer" containerID="9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe" Jul 06 22:53:49.980800 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.980779 2578 scope.go:117] "RemoveContainer" containerID="5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33" Jul 06 22:53:49.981065 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:53:49.981047 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33\": container with ID starting with 5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33 not found: ID does not exist" containerID="5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33" Jul 06 22:53:49.981158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.981072 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33"} err="failed to get container status \"5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33\": rpc error: code = NotFound desc = could not find container \"5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33\": container with ID starting with 5b31a5de5c522085c5bd4da4d059d57b22b4a46dd94c596fb33f183f22310f33 not found: ID does not exist" Jul 06 22:53:49.981158 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.981089 2578 scope.go:117] "RemoveContainer" containerID="9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe" Jul 06 22:53:49.981353 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:53:49.981335 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe\": container with ID starting with 9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe not found: ID does not exist" containerID="9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe" Jul 06 22:53:49.981394 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.981360 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe"} err="failed to get container status \"9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe\": rpc error: code = NotFound desc = could not find container \"9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe\": container with ID starting with 9056728ce337bbd859943f25af70e2a3c9263865abb512d446cf51ac32072afe not found: ID does not exist" Jul 06 22:53:49.986657 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.986631 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9"] Jul 06 22:53:49.998704 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:49.998674 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8f36f0-openshift-default-dfc666f8f-f4xr9"] Jul 06 22:53:50.379221 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:50.379182 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" path="/var/lib/kubelet/pods/e2f102fb-f391-47cc-8a2b-b60c94478a18/volumes" Jul 06 22:53:52.005474 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.005440 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv"] Jul 06 22:53:52.005932 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.005637 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" containerID="cri-o://543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b" gracePeriod=30 Jul 06 22:53:52.909575 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.909526 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx"] Jul 06 22:53:52.920888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.919339 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" Jul 06 22:53:52.920888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.919375 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" Jul 06 22:53:52.920888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.919402 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" Jul 06 22:53:52.920888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.919411 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" Jul 06 22:53:52.920888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.919427 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" Jul 06 22:53:52.920888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.919436 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" Jul 06 22:53:52.920888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.919535 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be79e6b-adc7-4dd1-813b-5987a74bafeb" containerName="istio-proxy" Jul 06 22:53:52.920888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.919546 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" Jul 06 22:53:52.920888 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.919556 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2f102fb-f391-47cc-8a2b-b60c94478a18" containerName="istio-proxy" Jul 06 22:53:52.927134 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.925098 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:52.928311 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.928284 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-2ef6a3-openshift-default-dockercfg-tgk9c\"" Jul 06 22:53:52.933133 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:52.933053 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx"] Jul 06 22:53:53.086771 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.086734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-data\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.087167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.086780 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-credential-socket\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.087167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.086817 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59560c21-5dce-409a-8bc0-5eed31116937-istio-podinfo\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.087167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.086853 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-envoy\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.087167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.086922 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-istio-token\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.087167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.086977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-socket\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.087167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.087006 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-certs\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.087167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.087027 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk4jd\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-kube-api-access-fk4jd\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.087167 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.087048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59560c21-5dce-409a-8bc0-5eed31116937-istiod-ca-cert\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.187992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.187894 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-data\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.187992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.187942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-credential-socket\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.187992 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.187963 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59560c21-5dce-409a-8bc0-5eed31116937-istio-podinfo\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188280 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188080 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-envoy\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188280 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188170 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-istio-token\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188280 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188202 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-socket\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188280 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188217 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-certs\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188280 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188247 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk4jd\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-kube-api-access-fk4jd\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188533 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188398 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-credential-socket\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188533 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59560c21-5dce-409a-8bc0-5eed31116937-istiod-ca-cert\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188643 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188562 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-socket\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188747 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188637 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-certs\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188747 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188663 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-data\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.188995 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.188976 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59560c21-5dce-409a-8bc0-5eed31116937-istiod-ca-cert\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.190885 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.190859 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-envoy\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.191081 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.191066 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59560c21-5dce-409a-8bc0-5eed31116937-istio-podinfo\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.197435 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.197410 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-istio-token\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.198182 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.198162 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk4jd\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-kube-api-access-fk4jd\") pod \"e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.245192 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.245153 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:53.391875 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.391848 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx"] Jul 06 22:53:53.393619 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:53:53.393577 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59560c21_5dce_409a_8bc0_5eed31116937.slice/crio-50535bca183ca5333b2785a87f556fc8ecddb957d9fb20c1fc6124a206c246c1 WatchSource:0}: Error finding container 50535bca183ca5333b2785a87f556fc8ecddb957d9fb20c1fc6124a206c246c1: Status 404 returned error can't find the container with id 50535bca183ca5333b2785a87f556fc8ecddb957d9fb20c1fc6124a206c246c1 Jul 06 22:53:53.395733 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.395705 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:53.395820 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.395770 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:53.395862 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.395841 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:53:53.988348 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.988313 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" event={"ID":"59560c21-5dce-409a-8bc0-5eed31116937","Type":"ContainerStarted","Data":"9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0"} Jul 06 22:53:53.988581 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:53.988368 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" event={"ID":"59560c21-5dce-409a-8bc0-5eed31116937","Type":"ContainerStarted","Data":"50535bca183ca5333b2785a87f556fc8ecddb957d9fb20c1fc6124a206c246c1"} Jul 06 22:53:54.013223 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:54.013170 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" podStartSLOduration=2.013153717 podStartE2EDuration="2.013153717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:53:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:54.011892618 +0000 UTC m=+2024.113620293" watchObservedRunningTime="2026-07-06 22:53:54.013153717 +0000 UTC m=+2024.114881376" Jul 06 22:53:54.245619 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:54.245523 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:54.250525 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:54.250501 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:54.994668 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:54.994629 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:54.995771 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:54.995748 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:53:57.255681 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.255653 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:57.427179 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427104 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkbvw\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-kube-api-access-tkbvw\") pod \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " Jul 06 22:53:57.427360 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427201 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-certs\") pod \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " Jul 06 22:53:57.427360 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427232 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-credential-socket\") pod \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " Jul 06 22:53:57.427360 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427250 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istiod-ca-cert\") pod \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " Jul 06 22:53:57.427360 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427298 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-socket\") pod \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " Jul 06 22:53:57.427561 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427437 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-data\") pod \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " Jul 06 22:53:57.427561 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427516 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-token\") pod \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " Jul 06 22:53:57.427561 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427529 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" (UID: "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:57.427561 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427538 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" (UID: "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:57.427561 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427554 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-podinfo\") pod \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " Jul 06 22:53:57.427798 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427554 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" (UID: "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:57.427798 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427585 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-envoy\") pod \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\" (UID: \"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b\") " Jul 06 22:53:57.427798 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427613 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" (UID: "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:53:57.427952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427880 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:57.427952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427897 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:57.427952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427911 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:57.427952 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.427926 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:57.428440 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.428414 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-data" (OuterVolumeSpecName: "istio-data") pod "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" (UID: "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:57.429620 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.429586 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-token" (OuterVolumeSpecName: "istio-token") pod "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" (UID: "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:57.429737 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.429655 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-kube-api-access-tkbvw" (OuterVolumeSpecName: "kube-api-access-tkbvw") pod "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" (UID: "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b"). InnerVolumeSpecName "kube-api-access-tkbvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:53:57.429966 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.429943 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" (UID: "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:53:57.430049 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.429960 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" (UID: "c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:53:57.528736 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.528697 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:57.528736 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.528731 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:57.528736 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.528742 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:57.529084 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.528752 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:57.529084 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:57.528768 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkbvw\" (UniqueName: \"kubernetes.io/projected/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b-kube-api-access-tkbvw\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 22:53:58.011489 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.011452 2578 generic.go:356] "Generic (PLEG): container finished" podID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerID="543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b" exitCode=0 Jul 06 22:53:58.011706 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.011508 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" Jul 06 22:53:58.011706 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.011540 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" event={"ID":"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b","Type":"ContainerDied","Data":"543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b"} Jul 06 22:53:58.011706 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.011599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv" event={"ID":"c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b","Type":"ContainerDied","Data":"bc2dd925c6efd41d1677a31e105324025150c263bcd470789487167eda532651"} Jul 06 22:53:58.011706 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.011614 2578 scope.go:117] "RemoveContainer" containerID="543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b" Jul 06 22:53:58.021496 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.021475 2578 scope.go:117] "RemoveContainer" containerID="70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24" Jul 06 22:53:58.030191 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.030169 2578 scope.go:117] "RemoveContainer" containerID="543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b" Jul 06 22:53:58.030477 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:53:58.030458 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b\": container with ID starting with 543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b not found: ID does not exist" containerID="543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b" Jul 06 22:53:58.030525 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.030486 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b"} err="failed to get container status \"543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b\": rpc error: code = NotFound desc = could not find container \"543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b\": container with ID starting with 543356bac0141b392263c40af7ed9912c7048090a426107688c03e2779dc7b0b not found: ID does not exist" Jul 06 22:53:58.030525 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.030503 2578 scope.go:117] "RemoveContainer" containerID="70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24" Jul 06 22:53:58.030775 ip-10-0-142-114 kubenswrapper[2578]: E0706 22:53:58.030751 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24\": container with ID starting with 70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24 not found: ID does not exist" containerID="70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24" Jul 06 22:53:58.030839 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.030783 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24"} err="failed to get container status \"70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24\": rpc error: code = NotFound desc = could not find container \"70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24\": container with ID starting with 70970685feeae55a07e5984f2ecfee3719dcf1987540e0c555d9432c7d813b24 not found: ID does not exist" Jul 06 22:53:58.038067 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.038045 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv"] Jul 06 22:53:58.046408 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.046382 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-164a1c-openshift-default-867bb7d9df-6bwnv"] Jul 06 22:53:58.379569 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:53:58.379540 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" path="/var/lib/kubelet/pods/c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b/volumes" Jul 06 22:54:01.081768 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.081723 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5"] Jul 06 22:54:01.082251 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.082133 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" Jul 06 22:54:01.082251 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.082149 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" Jul 06 22:54:01.082251 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.082231 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" Jul 06 22:54:01.082251 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.082243 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" Jul 06 22:54:01.086711 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.086691 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.089050 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.089026 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-6c153e-openshift-default-dockercfg-4fhrg\"" Jul 06 22:54:01.096520 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.096497 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5"] Jul 06 22:54:01.161085 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.161043 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-socket\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.161315 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.161106 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-certs\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.161315 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.161179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae03066-af8c-40c0-9157-69db91149556-istiod-ca-cert\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.161315 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.161239 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-envoy\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.161315 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.161274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l66cs\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-kube-api-access-l66cs\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.161540 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.161443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-istio-token\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.161540 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.161518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-data\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.161654 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.161551 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-credential-socket\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.161654 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.161588 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae03066-af8c-40c0-9157-69db91149556-istio-podinfo\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.262576 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.262533 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-data\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.262760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.262586 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-credential-socket\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.262760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.262624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae03066-af8c-40c0-9157-69db91149556-istio-podinfo\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.262760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.262663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-socket\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.262760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.262701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-certs\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.262760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.262736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae03066-af8c-40c0-9157-69db91149556-istiod-ca-cert\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.263041 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.262775 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-envoy\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.263041 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.262808 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l66cs\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-kube-api-access-l66cs\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.263041 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.262868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-istio-token\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.263041 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.263030 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-credential-socket\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.263277 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.263040 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-data\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.263549 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.263520 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-socket\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.263776 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.263748 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae03066-af8c-40c0-9157-69db91149556-istiod-ca-cert\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.263940 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.263831 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-certs\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.265562 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.265532 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-envoy\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.266206 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.266184 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae03066-af8c-40c0-9157-69db91149556-istio-podinfo\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.274680 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.274581 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-istio-token\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.275889 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.275861 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l66cs\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-kube-api-access-l66cs\") pod \"e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.398447 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.398326 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:01.559139 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.559097 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5"] Jul 06 22:54:01.561715 ip-10-0-142-114 kubenswrapper[2578]: W0706 22:54:01.561684 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae03066_af8c_40c0_9157_69db91149556.slice/crio-218acb88fac0bb758e207b4dccbd4c73be3a889bcf1894b9fb3e42ad9195f582 WatchSource:0}: Error finding container 218acb88fac0bb758e207b4dccbd4c73be3a889bcf1894b9fb3e42ad9195f582: Status 404 returned error can't find the container with id 218acb88fac0bb758e207b4dccbd4c73be3a889bcf1894b9fb3e42ad9195f582 Jul 06 22:54:01.563962 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.563921 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:54:01.564049 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.564013 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:54:01.564089 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:01.564054 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:54:02.039707 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:02.039675 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" event={"ID":"3ae03066-af8c-40c0-9157-69db91149556","Type":"ContainerStarted","Data":"cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc"} Jul 06 22:54:02.039909 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:02.039726 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" event={"ID":"3ae03066-af8c-40c0-9157-69db91149556","Type":"ContainerStarted","Data":"218acb88fac0bb758e207b4dccbd4c73be3a889bcf1894b9fb3e42ad9195f582"} Jul 06 22:54:02.065420 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:02.065351 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" podStartSLOduration=1.065334486 podStartE2EDuration="1.065334486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:54:02.063036502 +0000 UTC m=+2032.164764184" watchObservedRunningTime="2026-07-06 22:54:02.065334486 +0000 UTC m=+2032.167062216" Jul 06 22:54:02.398890 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:02.398853 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:02.404552 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:02.404518 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:03.044898 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:03.044872 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:03.045850 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:03.045827 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:09.996512 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:09.996472 2578 patch_prober.go:28] interesting pod/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 06 22:54:09.996929 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:09.996545 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 06 22:54:10.093260 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:10.093226 2578 generic.go:356] "Generic (PLEG): container finished" podID="59560c21-5dce-409a-8bc0-5eed31116937" containerID="9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0" exitCode=0 Jul 06 22:54:10.093430 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:10.093298 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" event={"ID":"59560c21-5dce-409a-8bc0-5eed31116937","Type":"ContainerDied","Data":"9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0"} Jul 06 22:54:10.093722 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:10.093708 2578 scope.go:117] "RemoveContainer" containerID="9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0" Jul 06 22:54:10.094701 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:10.094674 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:54:10.094760 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:10.094744 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:54:10.094798 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:10.094773 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:54:11.105827 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:11.105766 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" event={"ID":"59560c21-5dce-409a-8bc0-5eed31116937","Type":"ContainerStarted","Data":"64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025"} Jul 06 22:54:11.245661 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:11.245633 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:54:11.245925 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:11.245830 2578 patch_prober.go:28] interesting pod/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 06 22:54:11.245985 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:11.245960 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 06 22:54:12.246079 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:12.245974 2578 patch_prober.go:28] interesting pod/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 06 22:54:12.246079 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:12.246027 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 06 22:54:13.246174 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:13.246134 2578 patch_prober.go:28] interesting pod/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 06 22:54:13.246743 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:13.246193 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 06 22:54:14.245781 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:14.245744 2578 patch_prober.go:28] interesting pod/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 06 22:54:14.245972 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:14.245817 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 06 22:54:15.250297 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:15.250259 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:54:15.250742 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:15.250723 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:54:15.251378 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:15.251359 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 22:54:16.133332 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:16.133303 2578 generic.go:356] "Generic (PLEG): container finished" podID="3ae03066-af8c-40c0-9157-69db91149556" containerID="cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc" exitCode=0 Jul 06 22:54:16.133554 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:16.133391 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" event={"ID":"3ae03066-af8c-40c0-9157-69db91149556","Type":"ContainerDied","Data":"cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc"} Jul 06 22:54:16.133860 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:16.133843 2578 scope.go:117] "RemoveContainer" containerID="cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc" Jul 06 22:54:16.134789 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:16.134767 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:54:16.134858 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:16.134828 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:54:16.134898 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:16.134863 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 06 22:54:17.141607 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:17.141567 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" event={"ID":"3ae03066-af8c-40c0-9157-69db91149556","Type":"ContainerStarted","Data":"0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69"} Jul 06 22:54:17.399271 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:17.399168 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:17.400391 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:17.400362 2578 patch_prober.go:28] interesting pod/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 06 22:54:17.400532 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:17.400411 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 06 22:54:18.044980 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:18.044936 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:18.399441 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:18.399405 2578 patch_prober.go:28] interesting pod/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 06 22:54:18.399890 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:18.399461 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 06 22:54:19.398966 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:19.398929 2578 patch_prober.go:28] interesting pod/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 06 22:54:19.399176 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:19.398991 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 06 22:54:20.398908 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:20.398870 2578 patch_prober.go:28] interesting pod/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 06 22:54:20.399336 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:20.398927 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 06 22:54:21.403406 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:21.403372 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 22:54:21.404417 ip-10-0-142-114 kubenswrapper[2578]: I0706 22:54:21.404398 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 23:00:00.138180 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.138139 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722980-9fld7"] Jul 06 23:00:00.138840 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.138681 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" Jul 06 23:00:00.138840 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.138700 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c91a7d93-aeb4-4ac9-aaa6-48b9f1035d1b" containerName="istio-proxy" Jul 06 23:00:00.141889 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.141871 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" Jul 06 23:00:00.144573 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.144551 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-98x7w\"" Jul 06 23:00:00.156102 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.156066 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722980-9fld7"] Jul 06 23:00:00.215509 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.215464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv4fm\" (UniqueName: \"kubernetes.io/projected/ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e-kube-api-access-gv4fm\") pod \"maas-api-key-cleanup-29722980-9fld7\" (UID: \"ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e\") " pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" Jul 06 23:00:00.316367 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.316329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv4fm\" (UniqueName: \"kubernetes.io/projected/ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e-kube-api-access-gv4fm\") pod \"maas-api-key-cleanup-29722980-9fld7\" (UID: \"ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e\") " pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" Jul 06 23:00:00.326243 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.326217 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv4fm\" (UniqueName: \"kubernetes.io/projected/ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e-kube-api-access-gv4fm\") pod \"maas-api-key-cleanup-29722980-9fld7\" (UID: \"ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e\") " pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" Jul 06 23:00:00.452785 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.452694 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" Jul 06 23:00:00.591677 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.591645 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722980-9fld7"] Jul 06 23:00:00.593212 ip-10-0-142-114 kubenswrapper[2578]: W0706 23:00:00.593187 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba1d07fc_84ba_4fb3_bee7_7ec35cb4626e.slice/crio-600fad97c53c3abb225e4920c2db4a158aab3aee8c721649fac920f2189869f7 WatchSource:0}: Error finding container 600fad97c53c3abb225e4920c2db4a158aab3aee8c721649fac920f2189869f7: Status 404 returned error can't find the container with id 600fad97c53c3abb225e4920c2db4a158aab3aee8c721649fac920f2189869f7 Jul 06 23:00:00.974534 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.974493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" event={"ID":"ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e","Type":"ContainerStarted","Data":"6f94c88040dee0e111e0f7145b82dda9ca420ca72adc451b07f1823176004684"} Jul 06 23:00:00.974738 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.974563 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" event={"ID":"ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e","Type":"ContainerStarted","Data":"600fad97c53c3abb225e4920c2db4a158aab3aee8c721649fac920f2189869f7"} Jul 06 23:00:00.993144 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:00.993070 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" podStartSLOduration=0.993052352 podStartE2EDuration="993.052352ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:00:00.991635839 +0000 UTC m=+2391.093363502" watchObservedRunningTime="2026-07-06 23:00:00.993052352 +0000 UTC m=+2391.094780014" Jul 06 23:00:01.981992 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:01.981957 2578 generic.go:356] "Generic (PLEG): container finished" podID="ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e" containerID="6f94c88040dee0e111e0f7145b82dda9ca420ca72adc451b07f1823176004684" exitCode=0 Jul 06 23:00:01.982431 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:01.982051 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" event={"ID":"ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e","Type":"ContainerDied","Data":"6f94c88040dee0e111e0f7145b82dda9ca420ca72adc451b07f1823176004684"} Jul 06 23:00:03.126739 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:03.126709 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" Jul 06 23:00:03.244211 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:03.244093 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv4fm\" (UniqueName: \"kubernetes.io/projected/ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e-kube-api-access-gv4fm\") pod \"ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e\" (UID: \"ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e\") " Jul 06 23:00:03.246211 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:03.246178 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e-kube-api-access-gv4fm" (OuterVolumeSpecName: "kube-api-access-gv4fm") pod "ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e" (UID: "ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e"). InnerVolumeSpecName "kube-api-access-gv4fm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:00:03.345494 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:03.345459 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv4fm\" (UniqueName: \"kubernetes.io/projected/ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e-kube-api-access-gv4fm\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:03.994773 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:03.994742 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" Jul 06 23:00:03.994949 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:03.994789 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722980-9fld7" event={"ID":"ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e","Type":"ContainerDied","Data":"600fad97c53c3abb225e4920c2db4a158aab3aee8c721649fac920f2189869f7"} Jul 06 23:00:03.994949 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:03.994835 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="600fad97c53c3abb225e4920c2db4a158aab3aee8c721649fac920f2189869f7" Jul 06 23:00:53.282258 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:53.282167 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx"] Jul 06 23:00:53.282819 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:53.282437 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" containerID="cri-o://64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025" gracePeriod=30 Jul 06 23:00:58.540143 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.540101 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 23:00:58.646378 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646348 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-istio-token\") pod \"59560c21-5dce-409a-8bc0-5eed31116937\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " Jul 06 23:00:58.646599 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646422 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-certs\") pod \"59560c21-5dce-409a-8bc0-5eed31116937\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " Jul 06 23:00:58.646599 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646446 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59560c21-5dce-409a-8bc0-5eed31116937-istiod-ca-cert\") pod \"59560c21-5dce-409a-8bc0-5eed31116937\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " Jul 06 23:00:58.646599 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646493 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-envoy\") pod \"59560c21-5dce-409a-8bc0-5eed31116937\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " Jul 06 23:00:58.646778 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646632 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-data\") pod \"59560c21-5dce-409a-8bc0-5eed31116937\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " Jul 06 23:00:58.646778 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646685 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk4jd\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-kube-api-access-fk4jd\") pod \"59560c21-5dce-409a-8bc0-5eed31116937\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " Jul 06 23:00:58.646778 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646717 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59560c21-5dce-409a-8bc0-5eed31116937" (UID: "59560c21-5dce-409a-8bc0-5eed31116937"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:00:58.646778 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646739 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59560c21-5dce-409a-8bc0-5eed31116937-istio-podinfo\") pod \"59560c21-5dce-409a-8bc0-5eed31116937\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " Jul 06 23:00:58.646951 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646869 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-socket\") pod \"59560c21-5dce-409a-8bc0-5eed31116937\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " Jul 06 23:00:58.646951 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646915 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-credential-socket\") pod \"59560c21-5dce-409a-8bc0-5eed31116937\" (UID: \"59560c21-5dce-409a-8bc0-5eed31116937\") " Jul 06 23:00:58.646951 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.646916 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59560c21-5dce-409a-8bc0-5eed31116937-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59560c21-5dce-409a-8bc0-5eed31116937" (UID: "59560c21-5dce-409a-8bc0-5eed31116937"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:00:58.647159 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.647136 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59560c21-5dce-409a-8bc0-5eed31116937" (UID: "59560c21-5dce-409a-8bc0-5eed31116937"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:00:58.647440 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.647282 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59560c21-5dce-409a-8bc0-5eed31116937" (UID: "59560c21-5dce-409a-8bc0-5eed31116937"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:00:58.647440 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.647368 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:58.647440 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.647385 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59560c21-5dce-409a-8bc0-5eed31116937-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:58.647440 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.647397 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:58.647440 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.647411 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:58.647804 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.647476 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-data" (OuterVolumeSpecName: "istio-data") pod "59560c21-5dce-409a-8bc0-5eed31116937" (UID: "59560c21-5dce-409a-8bc0-5eed31116937"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:00:58.648841 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.648805 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59560c21-5dce-409a-8bc0-5eed31116937" (UID: "59560c21-5dce-409a-8bc0-5eed31116937"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:00:58.648950 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.648866 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-kube-api-access-fk4jd" (OuterVolumeSpecName: "kube-api-access-fk4jd") pod "59560c21-5dce-409a-8bc0-5eed31116937" (UID: "59560c21-5dce-409a-8bc0-5eed31116937"). InnerVolumeSpecName "kube-api-access-fk4jd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:00:58.648950 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.648888 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59560c21-5dce-409a-8bc0-5eed31116937-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59560c21-5dce-409a-8bc0-5eed31116937" (UID: "59560c21-5dce-409a-8bc0-5eed31116937"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:00:58.648950 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.648904 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-istio-token" (OuterVolumeSpecName: "istio-token") pod "59560c21-5dce-409a-8bc0-5eed31116937" (UID: "59560c21-5dce-409a-8bc0-5eed31116937"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:00:58.747892 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.747855 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:58.747892 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.747885 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:58.747892 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.747893 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59560c21-5dce-409a-8bc0-5eed31116937-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:58.747892 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.747902 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk4jd\" (UniqueName: \"kubernetes.io/projected/59560c21-5dce-409a-8bc0-5eed31116937-kube-api-access-fk4jd\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:58.748204 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:58.747911 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59560c21-5dce-409a-8bc0-5eed31116937-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:00:59.288915 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.288884 2578 generic.go:356] "Generic (PLEG): container finished" podID="59560c21-5dce-409a-8bc0-5eed31116937" containerID="64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025" exitCode=0 Jul 06 23:00:59.289091 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.288942 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" Jul 06 23:00:59.289091 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.288970 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" event={"ID":"59560c21-5dce-409a-8bc0-5eed31116937","Type":"ContainerDied","Data":"64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025"} Jul 06 23:00:59.289091 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.289028 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx" event={"ID":"59560c21-5dce-409a-8bc0-5eed31116937","Type":"ContainerDied","Data":"50535bca183ca5333b2785a87f556fc8ecddb957d9fb20c1fc6124a206c246c1"} Jul 06 23:00:59.289091 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.289058 2578 scope.go:117] "RemoveContainer" containerID="64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025" Jul 06 23:00:59.299173 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.299155 2578 scope.go:117] "RemoveContainer" containerID="9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0" Jul 06 23:00:59.308140 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.308107 2578 scope.go:117] "RemoveContainer" containerID="64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025" Jul 06 23:00:59.308384 ip-10-0-142-114 kubenswrapper[2578]: E0706 23:00:59.308364 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025\": container with ID starting with 64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025 not found: ID does not exist" containerID="64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025" Jul 06 23:00:59.308449 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.308391 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025"} err="failed to get container status \"64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025\": rpc error: code = NotFound desc = could not find container \"64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025\": container with ID starting with 64e53fee912851c85923ea582a7c388843857b940d5ecfcf694c0d1f1c0ec025 not found: ID does not exist" Jul 06 23:00:59.308449 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.308411 2578 scope.go:117] "RemoveContainer" containerID="9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0" Jul 06 23:00:59.308668 ip-10-0-142-114 kubenswrapper[2578]: E0706 23:00:59.308644 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0\": container with ID starting with 9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0 not found: ID does not exist" containerID="9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0" Jul 06 23:00:59.308760 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.308671 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0"} err="failed to get container status \"9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0\": rpc error: code = NotFound desc = could not find container \"9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0\": container with ID starting with 9336ce7499078f75aefc1ec1a0167aea9ed4b2d83309cf87ce627d830807afa0 not found: ID does not exist" Jul 06 23:00:59.313869 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.313845 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx"] Jul 06 23:00:59.319927 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:00:59.319902 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-2ef6a3-openshift-default-6bf97f99bc-9rztx"] Jul 06 23:01:00.380052 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:00.380019 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59560c21-5dce-409a-8bc0-5eed31116937" path="/var/lib/kubelet/pods/59560c21-5dce-409a-8bc0-5eed31116937/volumes" Jul 06 23:01:00.773396 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:00.773319 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5"] Jul 06 23:01:00.773548 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:00.773514 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" containerID="cri-o://0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69" gracePeriod=30 Jul 06 23:01:06.033726 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.033694 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 23:01:06.119828 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.119788 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l66cs\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-kube-api-access-l66cs\") pod \"3ae03066-af8c-40c0-9157-69db91149556\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " Jul 06 23:01:06.119828 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.119835 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-data\") pod \"3ae03066-af8c-40c0-9157-69db91149556\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " Jul 06 23:01:06.120096 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.119859 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-credential-socket\") pod \"3ae03066-af8c-40c0-9157-69db91149556\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " Jul 06 23:01:06.120096 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.119911 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-socket\") pod \"3ae03066-af8c-40c0-9157-69db91149556\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " Jul 06 23:01:06.120096 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.119925 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-envoy\") pod \"3ae03066-af8c-40c0-9157-69db91149556\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " Jul 06 23:01:06.120096 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.119947 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae03066-af8c-40c0-9157-69db91149556-istio-podinfo\") pod \"3ae03066-af8c-40c0-9157-69db91149556\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " Jul 06 23:01:06.120096 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.119976 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-istio-token\") pod \"3ae03066-af8c-40c0-9157-69db91149556\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " Jul 06 23:01:06.120395 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.120168 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-certs\") pod \"3ae03066-af8c-40c0-9157-69db91149556\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " Jul 06 23:01:06.120395 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.120227 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae03066-af8c-40c0-9157-69db91149556-istiod-ca-cert\") pod \"3ae03066-af8c-40c0-9157-69db91149556\" (UID: \"3ae03066-af8c-40c0-9157-69db91149556\") " Jul 06 23:01:06.120511 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.120488 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ae03066-af8c-40c0-9157-69db91149556" (UID: "3ae03066-af8c-40c0-9157-69db91149556"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:01:06.120776 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.120676 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ae03066-af8c-40c0-9157-69db91149556" (UID: "3ae03066-af8c-40c0-9157-69db91149556"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:01:06.120939 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.120663 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ae03066-af8c-40c0-9157-69db91149556" (UID: "3ae03066-af8c-40c0-9157-69db91149556"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:01:06.120939 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.120826 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ae03066-af8c-40c0-9157-69db91149556-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ae03066-af8c-40c0-9157-69db91149556" (UID: "3ae03066-af8c-40c0-9157-69db91149556"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:01:06.120939 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.120857 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ae03066-af8c-40c0-9157-69db91149556" (UID: "3ae03066-af8c-40c0-9157-69db91149556"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:01:06.122230 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.122196 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-kube-api-access-l66cs" (OuterVolumeSpecName: "kube-api-access-l66cs") pod "3ae03066-af8c-40c0-9157-69db91149556" (UID: "3ae03066-af8c-40c0-9157-69db91149556"). InnerVolumeSpecName "kube-api-access-l66cs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:01:06.122427 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.122404 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ae03066-af8c-40c0-9157-69db91149556-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ae03066-af8c-40c0-9157-69db91149556" (UID: "3ae03066-af8c-40c0-9157-69db91149556"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 23:01:06.122704 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.122683 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ae03066-af8c-40c0-9157-69db91149556" (UID: "3ae03066-af8c-40c0-9157-69db91149556"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:01:06.122704 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.122692 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ae03066-af8c-40c0-9157-69db91149556" (UID: "3ae03066-af8c-40c0-9157-69db91149556"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:01:06.149057 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.149025 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mglnv_31538d2a-93f9-4dee-bccf-6aec3dfa0e0c/manager/0.log" Jul 06 23:01:06.220975 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.220937 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-certs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:01:06.220975 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.220967 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ae03066-af8c-40c0-9157-69db91149556-istiod-ca-cert\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:01:06.220975 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.220976 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l66cs\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-kube-api-access-l66cs\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:01:06.221242 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.220988 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-data\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:01:06.221242 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.220998 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-credential-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:01:06.221242 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.221006 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-workload-socket\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:01:06.221242 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.221016 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ae03066-af8c-40c0-9157-69db91149556-istio-envoy\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:01:06.221242 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.221025 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ae03066-af8c-40c0-9157-69db91149556-istio-podinfo\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:01:06.221242 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.221032 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ae03066-af8c-40c0-9157-69db91149556-istio-token\") on node \"ip-10-0-142-114.ec2.internal\" DevicePath \"\"" Jul 06 23:01:06.278193 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.278156 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rdznz_9037b453-a672-4ba4-9a9b-845a773ade2c/manager/0.log" Jul 06 23:01:06.330192 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.330149 2578 generic.go:356] "Generic (PLEG): container finished" podID="3ae03066-af8c-40c0-9157-69db91149556" containerID="0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69" exitCode=0 Jul 06 23:01:06.330192 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.330195 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" Jul 06 23:01:06.330451 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.330207 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" event={"ID":"3ae03066-af8c-40c0-9157-69db91149556","Type":"ContainerDied","Data":"0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69"} Jul 06 23:01:06.330451 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.330257 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5" event={"ID":"3ae03066-af8c-40c0-9157-69db91149556","Type":"ContainerDied","Data":"218acb88fac0bb758e207b4dccbd4c73be3a889bcf1894b9fb3e42ad9195f582"} Jul 06 23:01:06.330451 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.330281 2578 scope.go:117] "RemoveContainer" containerID="0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69" Jul 06 23:01:06.339713 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.339694 2578 scope.go:117] "RemoveContainer" containerID="cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc" Jul 06 23:01:06.348302 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.348280 2578 scope.go:117] "RemoveContainer" containerID="0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69" Jul 06 23:01:06.348575 ip-10-0-142-114 kubenswrapper[2578]: E0706 23:01:06.348554 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69\": container with ID starting with 0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69 not found: ID does not exist" containerID="0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69" Jul 06 23:01:06.348648 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.348583 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69"} err="failed to get container status \"0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69\": rpc error: code = NotFound desc = could not find container \"0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69\": container with ID starting with 0bacd9a072f9b24ee74183795a4f770a701aa1b73be2fb18e8c2b5b20c446f69 not found: ID does not exist" Jul 06 23:01:06.348648 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.348602 2578 scope.go:117] "RemoveContainer" containerID="cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc" Jul 06 23:01:06.348884 ip-10-0-142-114 kubenswrapper[2578]: E0706 23:01:06.348867 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc\": container with ID starting with cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc not found: ID does not exist" containerID="cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc" Jul 06 23:01:06.348925 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.348890 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc"} err="failed to get container status \"cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc\": rpc error: code = NotFound desc = could not find container \"cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc\": container with ID starting with cd76412e8d7c0ba0dde79a51df9c751354f24fe189b66ed0dcdd8d96e45d04bc not found: ID does not exist" Jul 06 23:01:06.355625 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.355602 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5"] Jul 06 23:01:06.360917 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.360891 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-6c153e-openshift-default-5fc89d6487-542c5"] Jul 06 23:01:06.379258 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.379188 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ae03066-af8c-40c0-9157-69db91149556" path="/var/lib/kubelet/pods/3ae03066-af8c-40c0-9157-69db91149556/volumes" Jul 06 23:01:06.615170 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.615136 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722965-gl249_cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88/cleanup/0.log" Jul 06 23:01:06.732074 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.731990 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722980-9fld7_ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e/cleanup/0.log" Jul 06 23:01:06.964856 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:06.964825 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gd744_07eb0e43-b3b8-4a30-9337-ab887649d05f/server/0.log" Jul 06 23:01:07.085611 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:07.085586 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nl2k4_025fc94b-f743-42ef-a475-6adee1c6eaf3/manager/0.log" Jul 06 23:01:07.349640 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:07.349565 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c6597d756-5qfcm_fbc927a3-37e6-41ee-bb97-9f1283fff37c/copy-manifests/0.log" Jul 06 23:01:07.370050 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:07.370022 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c6597d756-5qfcm_fbc927a3-37e6-41ee-bb97-9f1283fff37c/manager/0.log" Jul 06 23:01:07.629886 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:07.629811 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-hjp5x_2e35f3a7-c841-47db-86a3-0d18a6924512/postgres/0.log" Jul 06 23:01:08.450616 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.450585 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz_0a69ad95-6583-444c-add0-abaf3f90c01a/util/0.log" Jul 06 23:01:08.460427 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.460400 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz_0a69ad95-6583-444c-add0-abaf3f90c01a/pull/0.log" Jul 06 23:01:08.469601 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.469571 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz_0a69ad95-6583-444c-add0-abaf3f90c01a/extract/0.log" Jul 06 23:01:08.583359 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.583332 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw_903ade62-f582-4c3f-b698-b69b9ada17b8/util/0.log" Jul 06 23:01:08.591654 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.591625 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw_903ade62-f582-4c3f-b698-b69b9ada17b8/pull/0.log" Jul 06 23:01:08.600996 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.600971 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw_903ade62-f582-4c3f-b698-b69b9ada17b8/extract/0.log" Jul 06 23:01:08.718538 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.718444 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6_5142f95c-b34c-47b2-8fbf-f9ad73f39db9/util/0.log" Jul 06 23:01:08.726826 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.726798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6_5142f95c-b34c-47b2-8fbf-f9ad73f39db9/pull/0.log" Jul 06 23:01:08.743372 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.743349 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6_5142f95c-b34c-47b2-8fbf-f9ad73f39db9/extract/0.log" Jul 06 23:01:08.862910 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.862876 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs_5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6/extract/0.log" Jul 06 23:01:08.871322 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.871297 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs_5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6/util/0.log" Jul 06 23:01:08.881536 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:08.881511 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs_5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6/pull/0.log" Jul 06 23:01:09.158639 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:09.158604 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h546x_9dd733e5-551a-4227-9fc5-6ffa7fa589f9/manager/0.log" Jul 06 23:01:09.281249 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:09.281204 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ktzht_425886c9-a8ad-45a6-be7f-dcfefb8ef3de/manager/0.log" Jul 06 23:01:09.399017 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:09.398991 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nx4bg_9ae7d4f4-56a7-4208-8c4f-1826f4699da5/kuadrant-console-plugin/0.log" Jul 06 23:01:09.524869 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:09.524780 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5dphw_96ea8dad-1100-463d-bbcc-d94d07b84603/registry-server/0.log" Jul 06 23:01:09.926360 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:09.926326 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vg9pr_bc1b9d49-257d-448f-8ddb-99b6450362c2/manager/0.log" Jul 06 23:01:10.658351 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:10.658324 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7588b845d6-lq682_32aefe50-699f-4ed3-9ce5-576d68543567/kube-auth-proxy/0.log" Jul 06 23:01:10.784147 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:10.784101 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ddkkx_a677d65b-4c54-498f-9514-d94a9d760563/istio-proxy/1.log" Jul 06 23:01:10.930324 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:10.930238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-q2df4_8c2a0878-3414-4c47-a3d4-502f8ded2a4b/payload-pre-processing/0.log" Jul 06 23:01:11.653814 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:11.653783 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-nzrfd_62400b3f-8bd9-497c-b400-a794e6c98b08/main/0.log" Jul 06 23:01:11.885798 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:11.885763 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbffx9k_34592bc9-3c31-4bf5-a336-2e4b0a7565c2/main/0.log" Jul 06 23:01:12.123069 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:12.123042 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-w4dtz_aa4722e2-290d-4501-a223-7edf983f60fe/main/0.log" Jul 06 23:01:16.321828 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.321789 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-59pq4/must-gather-9wrtc"] Jul 06 23:01:16.322329 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322311 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" Jul 06 23:01:16.322402 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322334 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" Jul 06 23:01:16.322402 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322345 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e" containerName="cleanup" Jul 06 23:01:16.322402 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322353 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e" containerName="cleanup" Jul 06 23:01:16.322402 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322394 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" Jul 06 23:01:16.322402 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322403 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" Jul 06 23:01:16.322685 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322413 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" Jul 06 23:01:16.322685 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322422 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" Jul 06 23:01:16.322685 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322510 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" Jul 06 23:01:16.322685 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322522 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e" containerName="cleanup" Jul 06 23:01:16.322685 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322532 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" Jul 06 23:01:16.322685 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322543 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" Jul 06 23:01:16.322685 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322645 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" Jul 06 23:01:16.322685 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322656 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="59560c21-5dce-409a-8bc0-5eed31116937" containerName="istio-proxy" Jul 06 23:01:16.323104 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.322741 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ae03066-af8c-40c0-9157-69db91149556" containerName="istio-proxy" Jul 06 23:01:16.327317 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.327295 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59pq4/must-gather-9wrtc" Jul 06 23:01:16.338301 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.338280 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-59pq4\"/\"default-dockercfg-zdbs2\"" Jul 06 23:01:16.339512 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.339494 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59pq4\"/\"openshift-service-ca.crt\"" Jul 06 23:01:16.342390 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.342369 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59pq4\"/\"kube-root-ca.crt\"" Jul 06 23:01:16.347236 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.347214 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59pq4/must-gather-9wrtc"] Jul 06 23:01:16.507741 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.507699 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ab9d0a41-2bce-41ca-a127-338a6f370916-must-gather-output\") pod \"must-gather-9wrtc\" (UID: \"ab9d0a41-2bce-41ca-a127-338a6f370916\") " pod="openshift-must-gather-59pq4/must-gather-9wrtc" Jul 06 23:01:16.507741 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.507740 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg95n\" (UniqueName: \"kubernetes.io/projected/ab9d0a41-2bce-41ca-a127-338a6f370916-kube-api-access-sg95n\") pod \"must-gather-9wrtc\" (UID: \"ab9d0a41-2bce-41ca-a127-338a6f370916\") " pod="openshift-must-gather-59pq4/must-gather-9wrtc" Jul 06 23:01:16.609346 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.609310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ab9d0a41-2bce-41ca-a127-338a6f370916-must-gather-output\") pod \"must-gather-9wrtc\" (UID: \"ab9d0a41-2bce-41ca-a127-338a6f370916\") " pod="openshift-must-gather-59pq4/must-gather-9wrtc" Jul 06 23:01:16.609346 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.609349 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg95n\" (UniqueName: \"kubernetes.io/projected/ab9d0a41-2bce-41ca-a127-338a6f370916-kube-api-access-sg95n\") pod \"must-gather-9wrtc\" (UID: \"ab9d0a41-2bce-41ca-a127-338a6f370916\") " pod="openshift-must-gather-59pq4/must-gather-9wrtc" Jul 06 23:01:16.609729 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.609704 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ab9d0a41-2bce-41ca-a127-338a6f370916-must-gather-output\") pod \"must-gather-9wrtc\" (UID: \"ab9d0a41-2bce-41ca-a127-338a6f370916\") " pod="openshift-must-gather-59pq4/must-gather-9wrtc" Jul 06 23:01:16.620518 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.620490 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg95n\" (UniqueName: \"kubernetes.io/projected/ab9d0a41-2bce-41ca-a127-338a6f370916-kube-api-access-sg95n\") pod \"must-gather-9wrtc\" (UID: \"ab9d0a41-2bce-41ca-a127-338a6f370916\") " pod="openshift-must-gather-59pq4/must-gather-9wrtc" Jul 06 23:01:16.637426 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.637399 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59pq4/must-gather-9wrtc" Jul 06 23:01:16.775689 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.775664 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59pq4/must-gather-9wrtc"] Jul 06 23:01:16.777348 ip-10-0-142-114 kubenswrapper[2578]: W0706 23:01:16.777320 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab9d0a41_2bce_41ca_a127_338a6f370916.slice/crio-8bb71722ba7446e4e10c71295205ee764daf17ead4a0c4626eb28e9730dc66f4 WatchSource:0}: Error finding container 8bb71722ba7446e4e10c71295205ee764daf17ead4a0c4626eb28e9730dc66f4: Status 404 returned error can't find the container with id 8bb71722ba7446e4e10c71295205ee764daf17ead4a0c4626eb28e9730dc66f4 Jul 06 23:01:16.779138 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:16.779102 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:01:17.397588 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:17.397553 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59pq4/must-gather-9wrtc" event={"ID":"ab9d0a41-2bce-41ca-a127-338a6f370916","Type":"ContainerStarted","Data":"8bb71722ba7446e4e10c71295205ee764daf17ead4a0c4626eb28e9730dc66f4"} Jul 06 23:01:18.408356 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:18.408321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59pq4/must-gather-9wrtc" event={"ID":"ab9d0a41-2bce-41ca-a127-338a6f370916","Type":"ContainerStarted","Data":"e314f30449768b0a9827edba65554d2567baa6fd0d24ea8ac5368051d18c7273"} Jul 06 23:01:18.408852 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:18.408365 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59pq4/must-gather-9wrtc" event={"ID":"ab9d0a41-2bce-41ca-a127-338a6f370916","Type":"ContainerStarted","Data":"be3192216070fccb1786e4d9e10e1d4269cb142a4be4080a2b1069d546bdcc7d"} Jul 06 23:01:18.428197 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:18.428149 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-59pq4/must-gather-9wrtc" podStartSLOduration=1.657909243 podStartE2EDuration="2.428134167s" totalImagesPullingTime="770.224924ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:01:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:01:18.426443463 +0000 UTC m=+2468.528171127" watchObservedRunningTime="2026-07-06 23:01:18.428134167 +0000 UTC m=+2468.529861823" Jul 06 23:01:19.417423 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:19.417389 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wmpgs_d1befd3e-57bc-4d94-9563-ef9479a3d10c/global-pull-secret-syncer/0.log" Jul 06 23:01:19.509555 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:19.509521 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6n28g_7ddb25aa-2738-45d5-9b3a-5e4ed8e2eae0/konnectivity-agent/0.log" Jul 06 23:01:19.633768 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:19.633738 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-114.ec2.internal_7b2e27a26a564387fecc3ee91d617486/haproxy/0.log" Jul 06 23:01:24.337397 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.337348 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz_0a69ad95-6583-444c-add0-abaf3f90c01a/extract/0.log" Jul 06 23:01:24.366458 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.366422 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz_0a69ad95-6583-444c-add0-abaf3f90c01a/util/0.log" Jul 06 23:01:24.399929 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.399894 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596scdz_0a69ad95-6583-444c-add0-abaf3f90c01a/pull/0.log" Jul 06 23:01:24.437304 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.437270 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw_903ade62-f582-4c3f-b698-b69b9ada17b8/extract/0.log" Jul 06 23:01:24.473320 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.473257 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw_903ade62-f582-4c3f-b698-b69b9ada17b8/util/0.log" Jul 06 23:01:24.507469 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.507438 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0pdgtw_903ade62-f582-4c3f-b698-b69b9ada17b8/pull/0.log" Jul 06 23:01:24.538659 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.538629 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6_5142f95c-b34c-47b2-8fbf-f9ad73f39db9/extract/0.log" Jul 06 23:01:24.565166 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.565131 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6_5142f95c-b34c-47b2-8fbf-f9ad73f39db9/util/0.log" Jul 06 23:01:24.599453 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.599356 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m4cp6_5142f95c-b34c-47b2-8fbf-f9ad73f39db9/pull/0.log" Jul 06 23:01:24.635815 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.635762 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs_5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6/extract/0.log" Jul 06 23:01:24.667252 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.667217 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs_5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6/util/0.log" Jul 06 23:01:24.700024 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:24.699981 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1vsnqs_5f24e8ea-1db1-47fa-ad5e-08ebe84e54d6/pull/0.log" Jul 06 23:01:25.027692 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:25.027607 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h546x_9dd733e5-551a-4227-9fc5-6ffa7fa589f9/manager/0.log" Jul 06 23:01:25.075276 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:25.075240 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ktzht_425886c9-a8ad-45a6-be7f-dcfefb8ef3de/manager/0.log" Jul 06 23:01:25.105398 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:25.105366 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nx4bg_9ae7d4f4-56a7-4208-8c4f-1826f4699da5/kuadrant-console-plugin/0.log" Jul 06 23:01:25.156570 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:25.156536 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5dphw_96ea8dad-1100-463d-bbcc-d94d07b84603/registry-server/0.log" Jul 06 23:01:25.445848 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:25.445815 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vg9pr_bc1b9d49-257d-448f-8ddb-99b6450362c2/manager/0.log" Jul 06 23:01:27.661631 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:27.661594 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lzdb4_c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96/node-exporter/0.log" Jul 06 23:01:27.693797 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:27.693770 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lzdb4_c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96/kube-rbac-proxy/0.log" Jul 06 23:01:27.724823 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:27.724798 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lzdb4_c5f6e4ee-9c5d-49cd-a8f4-f8c3dcf4eb96/init-textfile/0.log" Jul 06 23:01:29.324040 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.323999 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2"] Jul 06 23:01:29.332379 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.332343 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.340056 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.340029 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2"] Jul 06 23:01:29.444158 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.444085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-proc\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.444361 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.444261 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dngb7\" (UniqueName: \"kubernetes.io/projected/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-kube-api-access-dngb7\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.444361 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.444311 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-sys\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.444361 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.444335 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-lib-modules\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.444529 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.444447 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-podres\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.545673 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.545608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-proc\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.545883 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.545701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dngb7\" (UniqueName: \"kubernetes.io/projected/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-kube-api-access-dngb7\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.545883 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.545743 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-sys\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.545883 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.545763 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-proc\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.545883 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.545770 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-lib-modules\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.545883 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.545861 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-sys\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.546196 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.545888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-podres\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.546196 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.545896 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-lib-modules\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.546196 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.545980 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-podres\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.556958 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.556922 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dngb7\" (UniqueName: \"kubernetes.io/projected/24f73e20-9360-45b4-8e20-3b2a9e9fc0a9-kube-api-access-dngb7\") pod \"perf-node-gather-daemonset-4klx2\" (UID: \"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9\") " pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.650562 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.650290 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:29.836077 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:29.836049 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2"] Jul 06 23:01:29.838389 ip-10-0-142-114 kubenswrapper[2578]: W0706 23:01:29.838364 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod24f73e20_9360_45b4_8e20_3b2a9e9fc0a9.slice/crio-54001d9be480a23f5ca23ee5eec90aa14e0c878bd21e5af08991bbc2f0611ffa WatchSource:0}: Error finding container 54001d9be480a23f5ca23ee5eec90aa14e0c878bd21e5af08991bbc2f0611ffa: Status 404 returned error can't find the container with id 54001d9be480a23f5ca23ee5eec90aa14e0c878bd21e5af08991bbc2f0611ffa Jul 06 23:01:30.520505 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:30.520459 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" event={"ID":"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9","Type":"ContainerStarted","Data":"84fb5a001e3b90bfd9084b142706bc92e401c9b4119375c10787cae970b9378b"} Jul 06 23:01:30.520962 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:30.520532 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" event={"ID":"24f73e20-9360-45b4-8e20-3b2a9e9fc0a9","Type":"ContainerStarted","Data":"54001d9be480a23f5ca23ee5eec90aa14e0c878bd21e5af08991bbc2f0611ffa"} Jul 06 23:01:30.520962 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:30.520563 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:30.541506 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:30.541421 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" podStartSLOduration=1.541394599 podStartE2EDuration="1.541394599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:01:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:01:30.539021708 +0000 UTC m=+2480.640749374" watchObservedRunningTime="2026-07-06 23:01:30.541394599 +0000 UTC m=+2480.643122263" Jul 06 23:01:32.832057 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:32.832025 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n2j6t_224f9e83-3e16-471f-9e66-1ef1b08343c4/dns/0.log" Jul 06 23:01:32.873323 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:32.873293 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n2j6t_224f9e83-3e16-471f-9e66-1ef1b08343c4/kube-rbac-proxy/0.log" Jul 06 23:01:33.039030 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:33.038997 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k7phn_281a0cf8-525d-4171-b0f3-8020a15bfd48/dns-node-resolver/0.log" Jul 06 23:01:33.792856 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:33.792824 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w9g5x_5517402c-0226-4bbb-8d38-0200eb62df31/node-ca/0.log" Jul 06 23:01:35.007697 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:35.007667 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7588b845d6-lq682_32aefe50-699f-4ed3-9ce5-576d68543567/kube-auth-proxy/0.log" Jul 06 23:01:35.043799 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:35.043754 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ddkkx_a677d65b-4c54-498f-9514-d94a9d760563/istio-proxy/0.log" Jul 06 23:01:35.103917 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:35.103857 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ddkkx_a677d65b-4c54-498f-9514-d94a9d760563/istio-proxy/1.log" Jul 06 23:01:35.195404 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:35.195369 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-q2df4_8c2a0878-3414-4c47-a3d4-502f8ded2a4b/payload-pre-processing/0.log" Jul 06 23:01:36.032810 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:36.032780 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-285wd_e2a31111-3a9d-4f49-87fd-f0f417156ee7/serve-healthcheck-canary/0.log" Jul 06 23:01:36.543043 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:36.543013 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-59pq4/perf-node-gather-daemonset-4klx2" Jul 06 23:01:36.748571 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:36.748539 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9vz26_02417ea4-0e61-4d6e-9423-487aa1f2a053/kube-rbac-proxy/0.log" Jul 06 23:01:36.774761 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:36.774732 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9vz26_02417ea4-0e61-4d6e-9423-487aa1f2a053/exporter/0.log" Jul 06 23:01:36.802520 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:36.802422 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9vz26_02417ea4-0e61-4d6e-9423-487aa1f2a053/extractor/0.log" Jul 06 23:01:39.433612 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:39.433574 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mglnv_31538d2a-93f9-4dee-bccf-6aec3dfa0e0c/manager/0.log" Jul 06 23:01:39.697221 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:39.697183 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rdznz_9037b453-a672-4ba4-9a9b-845a773ade2c/manager/0.log" Jul 06 23:01:39.798003 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:39.797977 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722965-gl249_cfa4cae5-b6e3-4dd9-bd8d-9c0f748c6b88/cleanup/0.log" Jul 06 23:01:39.829038 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:39.829001 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722980-9fld7_ba1d07fc-84ba-4fb3-bee7-7ec35cb4626e/cleanup/0.log" Jul 06 23:01:39.984727 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:39.984617 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gd744_07eb0e43-b3b8-4a30-9337-ab887649d05f/server/0.log" Jul 06 23:01:40.030010 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:40.029982 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7588b845d6-lq682_32aefe50-699f-4ed3-9ce5-576d68543567/kube-auth-proxy/0.log" Jul 06 23:01:40.056157 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:40.056097 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ddkkx_a677d65b-4c54-498f-9514-d94a9d760563/istio-proxy/0.log" Jul 06 23:01:40.126599 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:40.126560 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nl2k4_025fc94b-f743-42ef-a475-6adee1c6eaf3/manager/0.log" Jul 06 23:01:40.138537 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:40.138510 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ddkkx_a677d65b-4c54-498f-9514-d94a9d760563/istio-proxy/1.log" Jul 06 23:01:40.169758 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:40.169728 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-q2df4_8c2a0878-3414-4c47-a3d4-502f8ded2a4b/payload-pre-processing/0.log" Jul 06 23:01:40.292865 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:40.292779 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c6597d756-5qfcm_fbc927a3-37e6-41ee-bb97-9f1283fff37c/manager/0.log" Jul 06 23:01:40.315261 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:40.315231 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c6597d756-5qfcm_fbc927a3-37e6-41ee-bb97-9f1283fff37c/copy-manifests/0.log" Jul 06 23:01:40.413985 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:40.413941 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-hjp5x_2e35f3a7-c841-47db-86a3-0d18a6924512/postgres/0.log" Jul 06 23:01:41.700390 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:41.700359 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7588b845d6-lq682_32aefe50-699f-4ed3-9ce5-576d68543567/kube-auth-proxy/0.log" Jul 06 23:01:41.728684 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:41.728651 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ddkkx_a677d65b-4c54-498f-9514-d94a9d760563/istio-proxy/0.log" Jul 06 23:01:41.784518 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:41.784426 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-ddkkx_a677d65b-4c54-498f-9514-d94a9d760563/istio-proxy/1.log" Jul 06 23:01:41.819663 ip-10-0-142-114 kubenswrapper[2578]: I0706 23:01:41.819630 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-q2df4_8c2a0878-3414-4c47-a3d4-502f8ded2a4b/payload-pre-processing/0.log"