Aug 03 16:02:13.662672 ip-10-0-142-140 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Aug 03 16:02:13.662687 ip-10-0-142-140 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Aug 03 16:02:13.662698 ip-10-0-142-140 systemd[1]: kubelet.service: Failed with result 'resources'. Aug 03 16:02:13.663037 ip-10-0-142-140 systemd[1]: Failed to start Kubernetes Kubelet. Aug 03 16:02:23.677727 ip-10-0-142-140 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Aug 03 16:02:23.677741 ip-10-0-142-140 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ec82c41d2e0e49bf8b85e200269a97b3 -- Aug 03 16:04:21.324946 ip-10-0-142-140 systemd[1]: Starting Kubernetes Kubelet... Aug 03 16:04:21.725539 ip-10-0-142-140 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 03 16:04:21.725539 ip-10-0-142-140 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 03 16:04:21.725539 ip-10-0-142-140 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 03 16:04:21.725539 ip-10-0-142-140 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 03 16:04:21.725539 ip-10-0-142-140 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 03 16:04:21.726527 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.726412 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 03 16:04:21.730677 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730660 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 03 16:04:21.730677 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730677 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730682 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730685 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730688 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730691 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730693 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730696 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730699 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730701 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730704 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730706 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730709 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730712 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730714 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730717 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730720 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730723 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730725 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730728 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730731 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 03 16:04:21.730748 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730733 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730736 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730738 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730741 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730744 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730746 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730749 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730752 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730754 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730757 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730760 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730762 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730765 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730768 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730772 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730775 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730778 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730781 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730784 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 03 16:04:21.731213 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730787 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730789 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730792 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730794 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730797 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730800 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730803 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730805 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730808 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730811 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730814 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730816 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730819 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730821 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730824 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730828 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730831 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730833 2582 feature_gate.go:328] unrecognized feature gate: Example2 Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730836 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730838 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 03 16:04:21.731693 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730841 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730844 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730847 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730849 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730852 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730854 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730857 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730862 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730866 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730869 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730871 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730874 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730877 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730880 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730882 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730885 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730887 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730890 2582 feature_gate.go:328] unrecognized feature gate: Example Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730892 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 03 16:04:21.732162 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730895 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730898 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730900 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730903 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730905 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730907 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730910 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730913 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730915 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730918 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730921 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730923 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730926 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730929 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730931 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730934 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730936 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730939 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730942 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.730944 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 03 16:04:21.732608 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731681 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731688 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731691 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731694 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731697 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731706 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731709 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731712 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731715 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731717 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731719 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731722 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731725 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731729 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731733 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731735 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731738 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731740 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731743 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 03 16:04:21.733092 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731746 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731749 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731751 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731754 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731756 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731759 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731761 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731764 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731767 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731770 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731772 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731775 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731777 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731780 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731783 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731785 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731788 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731790 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731792 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731802 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 03 16:04:21.733536 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731804 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731807 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731809 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731812 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731814 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731816 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731819 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731823 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731826 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731829 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731832 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731836 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731839 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731842 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731845 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731847 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731850 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731853 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731855 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731858 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 03 16:04:21.734018 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731863 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731867 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731870 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731873 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731876 2582 feature_gate.go:328] unrecognized feature gate: Example Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731878 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731881 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731883 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731886 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731888 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731891 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731893 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731896 2582 feature_gate.go:328] unrecognized feature gate: Example2 Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731899 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731901 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731905 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731907 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731910 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731912 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731915 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 03 16:04:21.734499 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731917 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731920 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731923 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731926 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731928 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731931 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731933 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731936 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731938 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731941 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731943 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731946 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731953 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731956 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731958 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731961 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731963 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731966 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731969 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 03 16:04:21.735019 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.731971 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732328 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732332 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732335 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732338 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732342 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732344 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732347 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732350 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732353 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732355 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732358 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732360 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732363 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732366 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732368 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732370 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732373 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732376 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732378 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 03 16:04:21.735456 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732381 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732383 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732386 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732388 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732390 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732394 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732396 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732399 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732401 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732403 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732406 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732408 2582 feature_gate.go:328] unrecognized feature gate: Example Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732411 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732413 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732415 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732418 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732420 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732423 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732426 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732429 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 03 16:04:21.735943 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732431 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732434 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732436 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732439 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732442 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732444 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732446 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732449 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732451 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732454 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732457 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732460 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732463 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732466 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732469 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732471 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732474 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732476 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732479 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 03 16:04:21.736449 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732481 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732483 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732486 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732488 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732492 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732495 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732498 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732501 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732503 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732506 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732508 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732511 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732513 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732516 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732518 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732521 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732525 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732528 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732531 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 03 16:04:21.736920 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732533 2582 feature_gate.go:328] unrecognized feature gate: Example2 Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732535 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732538 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732541 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732543 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732546 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732548 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732550 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732553 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732555 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732558 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732560 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732563 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732565 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732567 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732571 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732574 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732577 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732580 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732583 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 03 16:04:21.737413 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732586 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.732588 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732668 2582 flags.go:64] FLAG: --address="0.0.0.0" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732676 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732682 2582 flags.go:64] FLAG: --anonymous-auth="true" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732687 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732692 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732695 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732699 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732704 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732707 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732710 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732713 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732717 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732720 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732723 2582 flags.go:64] FLAG: --cgroup-root="" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732725 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732728 2582 flags.go:64] FLAG: --client-ca-file="" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732731 2582 flags.go:64] FLAG: --cloud-provider="external" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732734 2582 flags.go:64] FLAG: --cluster-dns="[]" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732738 2582 flags.go:64] FLAG: --cluster-domain="" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732741 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732744 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732747 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 03 16:04:21.737901 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732750 2582 flags.go:64] FLAG: --container-log-max-files="5" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732753 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732756 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732759 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732762 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732765 2582 flags.go:64] FLAG: --contention-profiling="false" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732768 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732771 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732775 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732778 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732782 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732785 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732788 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732790 2582 flags.go:64] FLAG: --enable-load-reader="false" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732793 2582 flags.go:64] FLAG: --enable-server="true" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732797 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732801 2582 flags.go:64] FLAG: --event-burst="100" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732805 2582 flags.go:64] FLAG: --event-qps="50" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732808 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732811 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732813 2582 flags.go:64] FLAG: --eviction-hard="" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732817 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732820 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732823 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732827 2582 flags.go:64] FLAG: --eviction-soft="" Aug 03 16:04:21.738454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732829 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732832 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732835 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732838 2582 flags.go:64] FLAG: --experimental-mounter-path="" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732841 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732844 2582 flags.go:64] FLAG: --fail-swap-on="true" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732847 2582 flags.go:64] FLAG: --feature-gates="" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732851 2582 flags.go:64] FLAG: --file-check-frequency="20s" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732854 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732857 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732860 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732863 2582 flags.go:64] FLAG: --healthz-port="10248" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732866 2582 flags.go:64] FLAG: --help="false" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732868 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-142-140.ec2.internal" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732872 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732877 2582 flags.go:64] FLAG: --http-check-frequency="20s" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732879 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732883 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732886 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732889 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732893 2582 flags.go:64] FLAG: --image-service-endpoint="" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732896 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732899 2582 flags.go:64] FLAG: --kube-api-burst="100" Aug 03 16:04:21.739323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732902 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732905 2582 flags.go:64] FLAG: --kube-api-qps="50" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732908 2582 flags.go:64] FLAG: --kube-reserved="" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732911 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732914 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732917 2582 flags.go:64] FLAG: --kubelet-cgroups="" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732920 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732923 2582 flags.go:64] FLAG: --lock-file="" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732925 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732928 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732931 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732937 2582 flags.go:64] FLAG: --log-json-split-stream="false" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732940 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732942 2582 flags.go:64] FLAG: --log-text-split-stream="false" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732945 2582 flags.go:64] FLAG: --logging-format="text" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732948 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732951 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732954 2582 flags.go:64] FLAG: --manifest-url="" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732957 2582 flags.go:64] FLAG: --manifest-url-header="" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732961 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732964 2582 flags.go:64] FLAG: --max-open-files="1000000" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732968 2582 flags.go:64] FLAG: --max-pods="110" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732971 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732974 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732978 2582 flags.go:64] FLAG: --memory-manager-policy="None" Aug 03 16:04:21.740346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732981 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732984 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732987 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732990 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.732997 2582 flags.go:64] FLAG: --node-status-max-images="50" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733000 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733003 2582 flags.go:64] FLAG: --oom-score-adj="-999" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733006 2582 flags.go:64] FLAG: --pod-cidr="" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733009 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733015 2582 flags.go:64] FLAG: --pod-manifest-path="" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733018 2582 flags.go:64] FLAG: --pod-max-pids="-1" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733021 2582 flags.go:64] FLAG: --pods-per-core="0" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733024 2582 flags.go:64] FLAG: --port="10250" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733027 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733030 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00956330cc8737064" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733033 2582 flags.go:64] FLAG: --qos-reserved="" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733036 2582 flags.go:64] FLAG: --read-only-port="10255" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733039 2582 flags.go:64] FLAG: --register-node="true" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733042 2582 flags.go:64] FLAG: --register-with-taints="" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733045 2582 flags.go:64] FLAG: --registry-burst="10" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733048 2582 flags.go:64] FLAG: --registry-qps="5" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733050 2582 flags.go:64] FLAG: --reserved-cpus="" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733053 2582 flags.go:64] FLAG: --reserved-memory="" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733057 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733060 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 03 16:04:21.741465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733062 2582 flags.go:64] FLAG: --rotate-certificates="false" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733065 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733068 2582 flags.go:64] FLAG: --runonce="false" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733071 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733074 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733077 2582 flags.go:64] FLAG: --seccomp-default="false" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733081 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733084 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733087 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733090 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733092 2582 flags.go:64] FLAG: --storage-driver-password="root" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733095 2582 flags.go:64] FLAG: --storage-driver-secure="false" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733101 2582 flags.go:64] FLAG: --storage-driver-table="stats" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733104 2582 flags.go:64] FLAG: --storage-driver-user="root" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733107 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733109 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733113 2582 flags.go:64] FLAG: --system-cgroups="" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733115 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733121 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733123 2582 flags.go:64] FLAG: --tls-cert-file="" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733126 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733130 2582 flags.go:64] FLAG: --tls-min-version="" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733132 2582 flags.go:64] FLAG: --tls-private-key-file="" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733135 2582 flags.go:64] FLAG: --topology-manager-policy="none" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733138 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 03 16:04:21.742578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733141 2582 flags.go:64] FLAG: --topology-manager-scope="container" Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733143 2582 flags.go:64] FLAG: --v="2" Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733147 2582 flags.go:64] FLAG: --version="false" Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733151 2582 flags.go:64] FLAG: --vmodule="" Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733155 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733158 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733279 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733283 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733286 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733289 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733292 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733295 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733297 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733301 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733304 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733306 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733309 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733311 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733314 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733319 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733322 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 03 16:04:21.743557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733325 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733327 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733330 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733333 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733336 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733339 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733341 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733343 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733346 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733349 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733351 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733355 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733359 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733361 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733364 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733367 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733369 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733372 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733374 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 03 16:04:21.744351 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733376 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733379 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733381 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733384 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733386 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733390 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733392 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733395 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733397 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733400 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733402 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733407 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733409 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733413 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733416 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733419 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733422 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733425 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733428 2582 feature_gate.go:328] unrecognized feature gate: Example Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733430 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 03 16:04:21.745097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733433 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733435 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733438 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733440 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733443 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733445 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733448 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733450 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733452 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733455 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733457 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733460 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733462 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733465 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733467 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733470 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733473 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733478 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733497 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 03 16:04:21.745596 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733501 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733504 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733506 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733509 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733513 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733516 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733519 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733522 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733525 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733528 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733530 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733533 2582 feature_gate.go:328] unrecognized feature gate: Example2 Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733536 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733539 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733542 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733545 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733548 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733551 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733553 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733556 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 03 16:04:21.746063 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733559 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733561 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733564 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733566 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733568 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.733571 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.733576 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.741478 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.741501 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.741527 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.741540 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741653 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741661 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741666 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 03 16:04:21.746560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741670 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741674 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741681 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741687 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741692 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741697 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741701 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741705 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741712 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741718 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741722 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741726 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741731 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741735 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741739 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741744 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741748 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741752 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 03 16:04:21.747053 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741756 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741760 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741765 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741769 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741773 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741777 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741782 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741786 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741799 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741804 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741808 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741812 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741817 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741821 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741825 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741830 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741834 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741838 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741842 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741846 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 03 16:04:21.747474 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741850 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741854 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741858 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741862 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741867 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741871 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741875 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741879 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741883 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741888 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741892 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741896 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741900 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741904 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741908 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741913 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741917 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741921 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741925 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741929 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 03 16:04:21.747961 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741934 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741945 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741950 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741954 2582 feature_gate.go:328] unrecognized feature gate: Example2 Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741958 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741962 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741967 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741970 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741974 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741978 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741982 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741986 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741991 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741995 2582 feature_gate.go:328] unrecognized feature gate: Example Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.741999 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742003 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742007 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742011 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742015 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742019 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742023 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 03 16:04:21.748439 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742027 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742032 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742037 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742041 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742045 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742049 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742053 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742057 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742061 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742065 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742070 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742076 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742082 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742094 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742098 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742102 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 03 16:04:21.748954 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742106 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.742114 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742338 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742346 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742351 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742355 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742359 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742363 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742367 2582 feature_gate.go:328] unrecognized feature gate: Example Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742372 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742379 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742384 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742390 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742393 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742398 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 03 16:04:21.749339 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742402 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742406 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742410 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742416 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742421 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742426 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742431 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742435 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742440 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742444 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742448 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742452 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742456 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742460 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742464 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742480 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742484 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742488 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742492 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742496 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 03 16:04:21.749713 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742501 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742504 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742509 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742513 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742517 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742521 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742525 2582 feature_gate.go:328] unrecognized feature gate: Example2 Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742531 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742537 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742542 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742546 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742550 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742554 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742558 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742562 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742566 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742570 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742574 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742577 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742581 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 03 16:04:21.750195 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742594 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742598 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742602 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742605 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742609 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742613 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742632 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742636 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742649 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742653 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742657 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742661 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742666 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742670 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742674 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742678 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742682 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742686 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742691 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742694 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 03 16:04:21.750696 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742699 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742703 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742707 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742711 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742715 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742719 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742723 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742726 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742731 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742735 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742739 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742744 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742748 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742753 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742757 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742761 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742765 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742771 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742776 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 03 16:04:21.751169 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742781 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742785 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742797 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742801 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742805 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742809 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:21.742813 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.742821 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.745235 2582 server.go:962] "Client rotation is on, will bootstrap in background" Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.749257 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.750398 2582 server.go:1019] "Starting client certificate rotation" Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.750493 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 03 16:04:21.751600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.750523 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 03 16:04:21.752676 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.752663 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 03 16:04:21.753795 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.753779 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 03 16:04:21.767665 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.767645 2582 log.go:25] "Validated CRI v1 runtime API" Aug 03 16:04:21.769703 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.769686 2582 log.go:25] "Validated CRI v1 image API" Aug 03 16:04:21.770410 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.770395 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 03 16:04:21.773132 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.773111 2582 fs.go:135] Filesystem UUIDs: map[45028c9e-2c0b-4acf-903b-81abfd0e6d2a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f8097e7b-35d0-4cb5-b3fa-8bddf573f48b:/dev/nvme0n1p4] Aug 03 16:04:21.773230 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.773130 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 03 16:04:21.778708 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.778566 2582 manager.go:216] Machine: {Timestamp:2026-08-03 16:04:21.777307049 +0000 UTC m=+0.335389625 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3220608 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20a922e4d69e8fd6c447d13bffc42a SystemUUID:ec20a922-e4d6-9e8f-d6c4-47d13bffc42a BootID:ec82c41d-2e0e-49bf-8b85-e200269a97b3 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:75:68:83:ee:6f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:75:68:83:ee:6f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:fc:5f:43:a0:82 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 03 16:04:21.778708 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.778699 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 03 16:04:21.778892 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.778856 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 03 16:04:21.782989 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.782958 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 03 16:04:21.783244 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.782990 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 03 16:04:21.783389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.783251 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-140.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 03 16:04:21.783921 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.783910 2582 topology_manager.go:138] "Creating topology manager with none policy" Aug 03 16:04:21.783952 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.783924 2582 container_manager_linux.go:309] "Creating device plugin manager" Aug 03 16:04:21.783952 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.783932 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 03 16:04:21.784595 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.784585 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 03 16:04:21.785449 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.785438 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 03 16:04:21.788700 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.788688 2582 state_mem.go:36] "Initialized new in-memory state store" Aug 03 16:04:21.788827 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.788818 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 03 16:04:21.788856 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.788849 2582 kubelet.go:486] "Attempting to sync node with API server" Aug 03 16:04:21.788892 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.788869 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 03 16:04:21.788922 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.788891 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 03 16:04:21.788922 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.788905 2582 kubelet.go:398] "Adding apiserver pod source" Aug 03 16:04:21.788922 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.788916 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 03 16:04:21.789364 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.789352 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 03 16:04:21.789423 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.789370 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 03 16:04:21.795845 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.795830 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 03 16:04:21.797137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.797120 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 03 16:04:21.797225 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.797158 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 03 16:04:21.797779 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.797754 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-140.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 03 16:04:21.797905 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.797889 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 03 16:04:21.798359 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798345 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 03 16:04:21.798425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798364 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 03 16:04:21.798425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798373 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 03 16:04:21.798425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798381 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 03 16:04:21.798425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798396 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 03 16:04:21.798425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798404 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 03 16:04:21.798425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798415 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 03 16:04:21.798425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798423 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 03 16:04:21.798687 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798442 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 03 16:04:21.798687 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798450 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 03 16:04:21.798687 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798463 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 03 16:04:21.798687 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.798471 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 03 16:04:21.799092 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.799081 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 03 16:04:21.799141 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.799095 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 03 16:04:21.802055 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.802039 2582 server.go:1268] "Started kubelet" Aug 03 16:04:21.802139 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.802106 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 03 16:04:21.802188 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.802119 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 03 16:04:21.802228 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.802190 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 03 16:04:21.802879 ip-10-0-142-140 systemd[1]: Started Kubernetes Kubelet. Aug 03 16:04:21.803237 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.803221 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 03 16:04:21.804017 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.803995 2582 server.go:310] "Adding debug handlers to kubelet server" Aug 03 16:04:21.808585 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.808568 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 03 16:04:21.808697 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.808684 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:21.809274 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.808866 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 03 16:04:21.809274 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809038 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-140.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 03 16:04:21.809400 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.809279 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 03 16:04:21.809509 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809489 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 03 16:04:21.809509 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809500 2582 factory.go:55] Registering systemd factory Aug 03 16:04:21.809610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809514 2582 factory.go:223] Registration of the systemd container factory successfully Aug 03 16:04:21.809610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809518 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 03 16:04:21.809610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809505 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 03 16:04:21.809759 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809694 2582 reconstruct.go:97] "Volume reconstruction finished" Aug 03 16:04:21.809759 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809705 2582 reconciler.go:29] "Reconciler: start to sync state" Aug 03 16:04:21.809759 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809730 2582 factory.go:153] Registering CRI-O factory Aug 03 16:04:21.809759 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809746 2582 factory.go:223] Registration of the crio container factory successfully Aug 03 16:04:21.809929 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809797 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 03 16:04:21.809929 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809819 2582 factory.go:103] Registering Raw factory Aug 03 16:04:21.809929 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.809833 2582 manager.go:1196] Started watching for new ooms in manager Aug 03 16:04:21.810170 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.810158 2582 manager.go:349] Starting recovery of all containers Aug 03 16:04:21.810708 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.809000 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-140.ec2.internal.18c85723662baf19 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-140.ec2.internal,UID:ip-10-0-142-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-140.ec2.internal,},FirstTimestamp:2026-08-03 16:04:21.802012441 +0000 UTC m=+0.360095018,LastTimestamp:2026-08-03 16:04:21.802012441 +0000 UTC m=+0.360095018,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-140.ec2.internal,}" Aug 03 16:04:21.813801 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.813772 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-140.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 03 16:04:21.813915 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.813894 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 03 16:04:21.822093 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.822074 2582 manager.go:354] Recovery completed Aug 03 16:04:21.823911 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.823879 2582 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Aug 03 16:04:21.826975 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.826963 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 03 16:04:21.828893 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.828856 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientMemory" Aug 03 16:04:21.828973 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.828909 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasNoDiskPressure" Aug 03 16:04:21.828973 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.828923 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientPID" Aug 03 16:04:21.829479 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.829465 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 03 16:04:21.829544 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.829480 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 03 16:04:21.829544 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.829509 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 03 16:04:21.829544 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.829527 2582 state_mem.go:36] "Initialized new in-memory state store" Aug 03 16:04:21.830196 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.830180 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientMemory" Aug 03 16:04:21.830277 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.830212 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasNoDiskPressure" Aug 03 16:04:21.830277 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.830226 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientPID" Aug 03 16:04:21.831088 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.831074 2582 policy_none.go:49] "None policy: Start" Aug 03 16:04:21.831088 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.831090 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 03 16:04:21.831198 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.831101 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 03 16:04:21.831374 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.831307 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-140.ec2.internal.18c8572367c5e7e7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-140.ec2.internal,UID:ip-10-0-142-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-140.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-140.ec2.internal,},FirstTimestamp:2026-08-03 16:04:21.828896743 +0000 UTC m=+0.386979319,LastTimestamp:2026-08-03 16:04:21.828896743 +0000 UTC m=+0.386979319,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-140.ec2.internal,}" Aug 03 16:04:21.832462 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.832449 2582 policy_none.go:47] "Start" Aug 03 16:04:21.839980 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.839909 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-140.ec2.internal.18c8572367c62dba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-140.ec2.internal,UID:ip-10-0-142-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-140.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-140.ec2.internal,},FirstTimestamp:2026-08-03 16:04:21.828914618 +0000 UTC m=+0.386997195,LastTimestamp:2026-08-03 16:04:21.828914618 +0000 UTC m=+0.386997195,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-140.ec2.internal,}" Aug 03 16:04:21.842467 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.842398 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-140.ec2.internal.18c8572367c66456 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-140.ec2.internal,UID:ip-10-0-142-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-140.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-140.ec2.internal,},FirstTimestamp:2026-08-03 16:04:21.828928598 +0000 UTC m=+0.387011177,LastTimestamp:2026-08-03 16:04:21.828928598 +0000 UTC m=+0.387011177,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-140.ec2.internal,}" Aug 03 16:04:21.852151 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.851941 2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-140.ec2.internal.18c8572367c5e7e7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-140.ec2.internal.18c8572367c5e7e7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-140.ec2.internal,UID:ip-10-0-142-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-140.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-140.ec2.internal,},FirstTimestamp:2026-08-03 16:04:21.828896743 +0000 UTC m=+0.386979319,LastTimestamp:2026-08-03 16:04:21.830196543 +0000 UTC m=+0.388279128,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-140.ec2.internal,}" Aug 03 16:04:21.861389 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.861321 2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-140.ec2.internal.18c8572367c62dba\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-140.ec2.internal.18c8572367c62dba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-140.ec2.internal,UID:ip-10-0-142-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-140.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-140.ec2.internal,},FirstTimestamp:2026-08-03 16:04:21.828914618 +0000 UTC m=+0.386997195,LastTimestamp:2026-08-03 16:04:21.83021893 +0000 UTC m=+0.388301507,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-140.ec2.internal,}" Aug 03 16:04:21.865299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.865284 2582 manager.go:337] "Starting Device Plugin manager" Aug 03 16:04:21.865393 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.865323 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 03 16:04:21.865393 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.865334 2582 server.go:87] "Starting device plugin registration server" Aug 03 16:04:21.865626 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.865614 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 03 16:04:21.865703 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.865627 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 03 16:04:21.865750 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.865717 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 03 16:04:21.865824 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.865805 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 03 16:04:21.865824 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.865820 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 03 16:04:21.866453 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.866435 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 03 16:04:21.866536 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.866493 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:21.870947 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.870869 2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-140.ec2.internal.18c8572367c66456\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-140.ec2.internal.18c8572367c66456 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-140.ec2.internal,UID:ip-10-0-142-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-140.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-140.ec2.internal,},FirstTimestamp:2026-08-03 16:04:21.828928598 +0000 UTC m=+0.387011177,LastTimestamp:2026-08-03 16:04:21.830232252 +0000 UTC m=+0.388314830,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-140.ec2.internal,}" Aug 03 16:04:21.879979 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.879910 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-140.ec2.internal.18c857236a1884c3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-140.ec2.internal,UID:ip-10-0-142-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-142-140.ec2.internal,},FirstTimestamp:2026-08-03 16:04:21.867865283 +0000 UTC m=+0.425947848,LastTimestamp:2026-08-03 16:04:21.867865283 +0000 UTC m=+0.425947848,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-140.ec2.internal,}" Aug 03 16:04:21.881970 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.881951 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c8lql" Aug 03 16:04:21.889871 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.889845 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c8lql" Aug 03 16:04:21.969726 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.969689 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 03 16:04:21.970927 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.970748 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientMemory" Aug 03 16:04:21.970927 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.970780 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasNoDiskPressure" Aug 03 16:04:21.970927 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.970791 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientPID" Aug 03 16:04:21.970927 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.970820 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-140.ec2.internal" Aug 03 16:04:21.984590 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.984542 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-140.ec2.internal" Aug 03 16:04:21.984590 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.984565 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-140.ec2.internal\": node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:21.987135 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.987102 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 03 16:04:21.988423 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.988402 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 03 16:04:21.988523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.988440 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 03 16:04:21.988523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.988459 2582 kubelet.go:2448] "Starting kubelet main sync loop" Aug 03 16:04:21.988523 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.988491 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 03 16:04:21.990073 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:21.990047 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 03 16:04:21.999505 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:21.999483 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:22.088831 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.088773 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal"] Aug 03 16:04:22.088992 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.088915 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 03 16:04:22.090000 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.089985 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientMemory" Aug 03 16:04:22.090059 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.090014 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasNoDiskPressure" Aug 03 16:04:22.090059 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.090027 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientPID" Aug 03 16:04:22.091382 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.091370 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 03 16:04:22.091492 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.091478 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.091530 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.091521 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 03 16:04:22.092302 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.092286 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientMemory" Aug 03 16:04:22.092357 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.092306 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientMemory" Aug 03 16:04:22.092357 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.092335 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasNoDiskPressure" Aug 03 16:04:22.092357 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.092346 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientPID" Aug 03 16:04:22.092455 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.092313 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasNoDiskPressure" Aug 03 16:04:22.092455 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.092407 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientPID" Aug 03 16:04:22.095506 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.095489 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.095592 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.095516 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 03 16:04:22.096238 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.096213 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientMemory" Aug 03 16:04:22.096238 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.096238 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasNoDiskPressure" Aug 03 16:04:22.096337 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.096249 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeHasSufficientPID" Aug 03 16:04:22.099966 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.099944 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:22.110992 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.110963 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f1e5cbdaea4f5cac69ceab27cf785688-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal\" (UID: \"f1e5cbdaea4f5cac69ceab27cf785688\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.111058 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.111001 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1e5cbdaea4f5cac69ceab27cf785688-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal\" (UID: \"f1e5cbdaea4f5cac69ceab27cf785688\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.111058 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.111021 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e7f1ea143023130aaaa7a2dabe0813a-config\") pod \"kube-apiserver-proxy-ip-10-0-142-140.ec2.internal\" (UID: \"6e7f1ea143023130aaaa7a2dabe0813a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.116385 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.116370 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-140.ec2.internal\" not found" node="ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.121152 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.121136 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-140.ec2.internal\" not found" node="ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.200927 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.200891 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:22.211331 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.211306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f1e5cbdaea4f5cac69ceab27cf785688-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal\" (UID: \"f1e5cbdaea4f5cac69ceab27cf785688\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.211404 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.211334 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1e5cbdaea4f5cac69ceab27cf785688-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal\" (UID: \"f1e5cbdaea4f5cac69ceab27cf785688\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.211404 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.211350 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e7f1ea143023130aaaa7a2dabe0813a-config\") pod \"kube-apiserver-proxy-ip-10-0-142-140.ec2.internal\" (UID: \"6e7f1ea143023130aaaa7a2dabe0813a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.211404 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.211384 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e7f1ea143023130aaaa7a2dabe0813a-config\") pod \"kube-apiserver-proxy-ip-10-0-142-140.ec2.internal\" (UID: \"6e7f1ea143023130aaaa7a2dabe0813a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.211514 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.211417 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f1e5cbdaea4f5cac69ceab27cf785688-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal\" (UID: \"f1e5cbdaea4f5cac69ceab27cf785688\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.211514 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.211420 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1e5cbdaea4f5cac69ceab27cf785688-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal\" (UID: \"f1e5cbdaea4f5cac69ceab27cf785688\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.301857 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.301772 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:22.408441 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.408395 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:22.419672 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.419653 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.424558 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.424121 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.511553 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.511506 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:22.615952 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.615871 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:22.716949 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.716911 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-140.ec2.internal\" not found" Aug 03 16:04:22.749597 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.749560 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 03 16:04:22.749597 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.749588 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 03 16:04:22.750170 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.749703 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 03 16:04:22.790193 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.790157 2582 apiserver.go:52] "Watching apiserver" Aug 03 16:04:22.801896 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.801863 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 03 16:04:22.802208 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.802185 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-n9756","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq","openshift-cluster-node-tuning-operator/tuned-6xxtl","openshift-dns/node-resolver-8g4rz","openshift-multus/network-metrics-daemon-2dgwx","openshift-ovn-kubernetes/ovnkube-node-5x55b","kube-system/global-pull-secret-syncer-rq67m","openshift-image-registry/node-ca-2vkng","openshift-multus/multus-additional-cni-plugins-lkckc","openshift-multus/multus-tqvvn","openshift-network-diagnostics/network-check-target-p65bb","openshift-network-operator/iptables-alerter-b7wwv"] Aug 03 16:04:22.803779 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.803763 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:22.804990 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.804968 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.806552 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.806534 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.807031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.807014 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 03 16:04:22.807129 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.807113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 03 16:04:22.807180 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.807113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gm6s2\"" Aug 03 16:04:22.807558 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.807545 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.807704 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.807685 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-glj2d\"" Aug 03 16:04:22.807799 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.807765 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 03 16:04:22.808597 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.808575 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 03 16:04:22.808696 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.808625 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 03 16:04:22.808860 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.808843 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.808915 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.808872 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:22.808951 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.808927 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:22.809184 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.809166 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 03 16:04:22.809315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.809300 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 03 16:04:22.809374 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.809315 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 03 16:04:22.809556 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.809537 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kzcvl\"" Aug 03 16:04:22.810133 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.810113 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.811071 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.811057 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nll8c\"" Aug 03 16:04:22.811135 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.811052 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 03 16:04:22.811295 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.811269 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:22.811409 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.811330 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:22.811485 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.811404 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 03 16:04:22.813399 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.812790 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:22.813646 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.813622 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ssntm\"" Aug 03 16:04:22.813836 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.813819 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 03 16:04:22.813907 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.813881 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 03 16:04:22.814299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.814281 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 03 16:04:22.815277 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.814495 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 03 16:04:22.815277 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.814824 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:22.815541 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.815511 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 03 16:04:22.815664 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.815646 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 03 16:04:22.816396 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.816376 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8skqn\"" Aug 03 16:04:22.816482 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.816432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.817591 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817574 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-socket-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.817657 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817599 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-modprobe-d\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.817657 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9a4751f-3a31-448f-a115-8d487b785137-hosts-file\") pod \"node-resolver-8g4rz\" (UID: \"b9a4751f-3a31-448f-a115-8d487b785137\") " pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.817657 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817647 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjk4k\" (UniqueName: \"kubernetes.io/projected/4baf391b-a641-4c8e-a1f0-e70a089ef564-kube-api-access-xjk4k\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:22.817844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817669 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-run-systemd\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.817844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817687 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-log-socket\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.817844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-run-ovn-kubernetes\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.817844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.817844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817803 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-slash\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.817844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817829 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-run-netns\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.817844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817845 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817882 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-kubernetes\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-sysctl-d\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817938 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-sysctl-conf\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817961 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa6b0903-c5d7-4063-a62e-770d5ed6a211-tmp\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.817989 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-systemd-units\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818023 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-cni-bin\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818033 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818055 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-ovnkube-script-lib\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxbdc\" (UniqueName: \"kubernetes.io/projected/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-kube-api-access-wxbdc\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.818096 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:22.818137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818104 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-systemd\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818169 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-run\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818206 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-tuned\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818237 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-etc-openvswitch\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818263 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-registration-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818305 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-kubelet\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818342 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818369 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a667877-9466-493b-beb0-fb6c194c11c9-konnectivity-ca\") pod \"konnectivity-agent-n9756\" (UID: \"3a667877-9466-493b-beb0-fb6c194c11c9\") " pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818395 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-sys-fs\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818418 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-sys\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818441 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-var-lib-openvswitch\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818474 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-run-openvswitch\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818511 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-cni-netd\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818548 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-ovnkube-config\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818582 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-device-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818604 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q52k7\" (UniqueName: \"kubernetes.io/projected/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-kube-api-access-q52k7\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.818658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818619 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-sysconfig\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818642 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-var-lib-kubelet\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818736 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-run-ovn\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818777 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-env-overrides\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818813 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-ovn-node-metrics-cert\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818833 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a667877-9466-493b-beb0-fb6c194c11c9-agent-certs\") pod \"konnectivity-agent-n9756\" (UID: \"3a667877-9466-493b-beb0-fb6c194c11c9\") " pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818862 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-lib-modules\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818878 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-host\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpnhp\" (UniqueName: \"kubernetes.io/projected/fa6b0903-c5d7-4063-a62e-770d5ed6a211-kube-api-access-rpnhp\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818913 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a4751f-3a31-448f-a115-8d487b785137-tmp-dir\") pod \"node-resolver-8g4rz\" (UID: \"b9a4751f-3a31-448f-a115-8d487b785137\") " pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818927 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mzh7\" (UniqueName: \"kubernetes.io/projected/b9a4751f-3a31-448f-a115-8d487b785137-kube-api-access-2mzh7\") pod \"node-resolver-8g4rz\" (UID: \"b9a4751f-3a31-448f-a115-8d487b785137\") " pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.819263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.818942 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-node-log\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.819744 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.819480 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:22.820184 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.820168 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 03 16:04:22.820469 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.820445 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 03 16:04:22.820469 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.820457 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kts2d\"" Aug 03 16:04:22.820645 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.820494 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 03 16:04:22.820645 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.820507 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 03 16:04:22.820645 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.820553 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 03 16:04:22.820645 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.820463 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 03 16:04:22.820645 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.820595 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" Aug 03 16:04:22.821000 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.820985 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal"] Aug 03 16:04:22.821389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.821370 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 03 16:04:22.821471 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.821452 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 03 16:04:22.821523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.821472 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 03 16:04:22.821739 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.821722 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v8l4s\"" Aug 03 16:04:22.821841 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.821824 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 03 16:04:22.822597 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.822579 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x4ksf\"" Aug 03 16:04:22.822689 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.822614 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 03 16:04:22.823214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.823202 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 03 16:04:22.823267 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.823203 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 03 16:04:22.829727 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.829710 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 03 16:04:22.833332 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.833316 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 03 16:04:22.833418 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.833400 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal"] Aug 03 16:04:22.843797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.843779 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6zz29" Aug 03 16:04:22.854753 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.854736 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6zz29" Aug 03 16:04:22.893782 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.893739 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-02 15:59:22 +0000 UTC" deadline="2028-05-01 23:40:44.246412144 +0000 UTC" Aug 03 16:04:22.893782 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.893781 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15295h36m21.352634777s" Aug 03 16:04:22.910400 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.910378 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 03 16:04:22.919793 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.919401 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-run-openvswitch\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.919793 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.919433 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-run-openvswitch\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.919793 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.919459 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q52k7\" (UniqueName: \"kubernetes.io/projected/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-kube-api-access-q52k7\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920130 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-sysconfig\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920177 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-var-lib-kubelet\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920215 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-multus-cni-dir\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-os-release\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920279 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-var-lib-cni-multus\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920314 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-run-ovn\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920380 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-ovn-node-metrics-cert\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920406 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a667877-9466-493b-beb0-fb6c194c11c9-agent-certs\") pod \"konnectivity-agent-n9756\" (UID: \"3a667877-9466-493b-beb0-fb6c194c11c9\") " pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:22.920461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.920435 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-host\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.921192 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.921171 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-sysconfig\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.921982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.921261 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-var-lib-kubelet\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.921982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.921287 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:22.921982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.921352 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:22.921982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.921437 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-run-ovn\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.921982 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.921542 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:22.921982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.921628 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-host\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.921982 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:22.921662 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs podName:4baf391b-a641-4c8e-a1f0-e70a089ef564 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:23.421603324 +0000 UTC m=+1.979685907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs") pod "network-metrics-daemon-2dgwx" (UID: "4baf391b-a641-4c8e-a1f0-e70a089ef564") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:22.921982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.921749 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 03 16:04:22.921982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.921960 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a4751f-3a31-448f-a115-8d487b785137-tmp-dir\") pod \"node-resolver-8g4rz\" (UID: \"b9a4751f-3a31-448f-a115-8d487b785137\") " pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922117 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mzh7\" (UniqueName: \"kubernetes.io/projected/b9a4751f-3a31-448f-a115-8d487b785137-kube-api-access-2mzh7\") pod \"node-resolver-8g4rz\" (UID: \"b9a4751f-3a31-448f-a115-8d487b785137\") " pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922157 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-node-log\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922219 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-node-log\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-socket-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922279 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-modprobe-d\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922315 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-system-cni-dir\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9a4751f-3a31-448f-a115-8d487b785137-hosts-file\") pod \"node-resolver-8g4rz\" (UID: \"b9a4751f-3a31-448f-a115-8d487b785137\") " pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922371 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjk4k\" (UniqueName: \"kubernetes.io/projected/4baf391b-a641-4c8e-a1f0-e70a089ef564-kube-api-access-xjk4k\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922403 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-run-ovn-kubernetes\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922413 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-socket-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922418 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9a4751f-3a31-448f-a115-8d487b785137-tmp-dir\") pod \"node-resolver-8g4rz\" (UID: \"b9a4751f-3a31-448f-a115-8d487b785137\") " pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922419 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-modprobe-d\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-slash\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922492 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b9a4751f-3a31-448f-a115-8d487b785137-hosts-file\") pod \"node-resolver-8g4rz\" (UID: \"b9a4751f-3a31-448f-a115-8d487b785137\") " pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922528 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-slash\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.922548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922550 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-kubernetes\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922589 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-sysctl-conf\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922619 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-run-ovn-kubernetes\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922652 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-kubernetes\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922658 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c443b634-9b93-4578-b47b-00a9cb4be95b-serviceca\") pod \"node-ca-2vkng\" (UID: \"c443b634-9b93-4578-b47b-00a9cb4be95b\") " pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922725 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922770 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-systemd-units\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922731 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-sysctl-conf\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922870 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-cni-bin\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922918 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-cni-bin\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922934 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-systemd-units\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-systemd\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.922974 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-run\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923008 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-tuned\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923043 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8xsn\" (UniqueName: \"kubernetes.io/projected/f1cce4c8-a3ed-4133-b6bd-834c3b52f037-kube-api-access-n8xsn\") pod \"iptables-alerter-b7wwv\" (UID: \"f1cce4c8-a3ed-4133-b6bd-834c3b52f037\") " pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923075 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c443b634-9b93-4578-b47b-00a9cb4be95b-host\") pod \"node-ca-2vkng\" (UID: \"c443b634-9b93-4578-b47b-00a9cb4be95b\") " pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:22.923577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923041 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-systemd\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923100 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-etc-openvswitch\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923152 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-etc-openvswitch\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923168 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-registration-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923229 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-multus-conf-dir\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923242 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-run\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923267 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923856 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-registration-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923872 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a667877-9466-493b-beb0-fb6c194c11c9-konnectivity-ca\") pod \"konnectivity-agent-n9756\" (UID: \"3a667877-9466-493b-beb0-fb6c194c11c9\") " pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923934 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-sys-fs\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.923966 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924005 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cghp5\" (UniqueName: \"kubernetes.io/projected/caa17516-2b14-491b-aa49-c53308141da3-kube-api-access-cghp5\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924037 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-var-lib-openvswitch\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924063 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-cni-netd\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924089 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-ovnkube-config\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924143 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-cni-netd\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924198 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-sys-fs\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.924483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924234 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-var-lib-openvswitch\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924276 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-device-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924315 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-system-cni-dir\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924340 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-hostroot\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-env-overrides\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924400 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-lib-modules\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpnhp\" (UniqueName: \"kubernetes.io/projected/fa6b0903-c5d7-4063-a62e-770d5ed6a211-kube-api-access-rpnhp\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924453 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a667877-9466-493b-beb0-fb6c194c11c9-konnectivity-ca\") pod \"konnectivity-agent-n9756\" (UID: \"3a667877-9466-493b-beb0-fb6c194c11c9\") " pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924461 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1cce4c8-a3ed-4133-b6bd-834c3b52f037-iptables-alerter-script\") pod \"iptables-alerter-b7wwv\" (UID: \"f1cce4c8-a3ed-4133-b6bd-834c3b52f037\") " pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924515 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924526 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-os-release\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924573 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-device-dir\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924599 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-cnibin\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924705 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-ovnkube-config\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924771 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/caa17516-2b14-491b-aa49-c53308141da3-multus-daemon-config\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924813 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16baf6d7-7c56-4d29-82bd-36c91c330974-dbus\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924842 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:22.925276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924868 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caa17516-2b14-491b-aa49-c53308141da3-cni-binary-copy\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924911 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-env-overrides\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924928 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-run-systemd\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924958 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-log-socket\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.924983 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925007 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-sysctl-d\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925033 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-run-netns\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925057 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa6b0903-c5d7-4063-a62e-770d5ed6a211-tmp\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1cce4c8-a3ed-4133-b6bd-834c3b52f037-host-slash\") pod \"iptables-alerter-b7wwv\" (UID: \"f1cce4c8-a3ed-4133-b6bd-834c3b52f037\") " pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925170 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsm7d\" (UniqueName: \"kubernetes.io/projected/c443b634-9b93-4578-b47b-00a9cb4be95b-kube-api-access-tsm7d\") pod \"node-ca-2vkng\" (UID: \"c443b634-9b93-4578-b47b-00a9cb4be95b\") " pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925212 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-cnibin\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925237 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-multus-socket-dir-parent\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925261 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-run-k8s-cni-cncf-io\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925287 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-ovnkube-script-lib\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925318 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxbdc\" (UniqueName: \"kubernetes.io/projected/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-kube-api-access-wxbdc\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925333 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-run-systemd\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925415 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-sysctl-d\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.926021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925435 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-run-netns\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925478 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925491 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-log-socket\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925509 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-run-netns\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925514 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-ovn-node-metrics-cert\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925528 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-lib-modules\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925550 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-etc-kubernetes\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925576 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-etc-selinux\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925578 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6smth\" (UniqueName: \"kubernetes.io/projected/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-kube-api-access-6smth\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925628 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-var-lib-cni-bin\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925646 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a667877-9466-493b-beb0-fb6c194c11c9-agent-certs\") pod \"konnectivity-agent-n9756\" (UID: \"3a667877-9466-493b-beb0-fb6c194c11c9\") " pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925666 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-var-lib-kubelet\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925717 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-kubelet\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925745 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-sys\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925756 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-host-kubelet\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925781 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-run-multus-certs\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925804 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa6b0903-c5d7-4063-a62e-770d5ed6a211-sys\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.926525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925838 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16baf6d7-7c56-4d29-82bd-36c91c330974-kubelet-config\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:22.927130 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.925994 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fa6b0903-c5d7-4063-a62e-770d5ed6a211-etc-tuned\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.927130 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.926107 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-ovnkube-script-lib\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.927471 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.927452 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa6b0903-c5d7-4063-a62e-770d5ed6a211-tmp\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:22.928830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.928809 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q52k7\" (UniqueName: \"kubernetes.io/projected/2150d1e5-3abc-47b1-a09a-37c8f04ab4ab-kube-api-access-q52k7\") pod \"aws-ebs-csi-driver-node-lnnnq\" (UID: \"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:22.931188 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.931163 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mzh7\" (UniqueName: \"kubernetes.io/projected/b9a4751f-3a31-448f-a115-8d487b785137-kube-api-access-2mzh7\") pod \"node-resolver-8g4rz\" (UID: \"b9a4751f-3a31-448f-a115-8d487b785137\") " pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:22.931505 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.931486 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjk4k\" (UniqueName: \"kubernetes.io/projected/4baf391b-a641-4c8e-a1f0-e70a089ef564-kube-api-access-xjk4k\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:22.937423 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.937398 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxbdc\" (UniqueName: \"kubernetes.io/projected/a3fec6fc-6b35-45f2-ba0e-e0d57481b446-kube-api-access-wxbdc\") pod \"ovnkube-node-5x55b\" (UID: \"a3fec6fc-6b35-45f2-ba0e-e0d57481b446\") " pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:22.937554 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:22.937526 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpnhp\" (UniqueName: \"kubernetes.io/projected/fa6b0903-c5d7-4063-a62e-770d5ed6a211-kube-api-access-rpnhp\") pod \"tuned-6xxtl\" (UID: \"fa6b0903-c5d7-4063-a62e-770d5ed6a211\") " pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:23.026267 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026232 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16baf6d7-7c56-4d29-82bd-36c91c330974-kubelet-config\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:23.026267 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026279 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-multus-cni-dir\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026302 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-os-release\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026326 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-var-lib-cni-multus\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026383 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16baf6d7-7c56-4d29-82bd-36c91c330974-kubelet-config\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026447 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-multus-cni-dir\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026461 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-os-release\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026483 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-var-lib-cni-multus\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026459 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-system-cni-dir\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.026523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026510 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-system-cni-dir\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026532 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c443b634-9b93-4578-b47b-00a9cb4be95b-serviceca\") pod \"node-ca-2vkng\" (UID: \"c443b634-9b93-4578-b47b-00a9cb4be95b\") " pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026551 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026578 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8xsn\" (UniqueName: \"kubernetes.io/projected/f1cce4c8-a3ed-4133-b6bd-834c3b52f037-kube-api-access-n8xsn\") pod \"iptables-alerter-b7wwv\" (UID: \"f1cce4c8-a3ed-4133-b6bd-834c3b52f037\") " pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026602 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c443b634-9b93-4578-b47b-00a9cb4be95b-host\") pod \"node-ca-2vkng\" (UID: \"c443b634-9b93-4578-b47b-00a9cb4be95b\") " pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026627 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026737 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-multus-conf-dir\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026784 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026786 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c443b634-9b93-4578-b47b-00a9cb4be95b-host\") pod \"node-ca-2vkng\" (UID: \"c443b634-9b93-4578-b47b-00a9cb4be95b\") " pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026850 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-multus-conf-dir\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cghp5\" (UniqueName: \"kubernetes.io/projected/caa17516-2b14-491b-aa49-c53308141da3-kube-api-access-cghp5\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026970 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-system-cni-dir\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.026995 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-hostroot\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027012 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c443b634-9b93-4578-b47b-00a9cb4be95b-serviceca\") pod \"node-ca-2vkng\" (UID: \"c443b634-9b93-4578-b47b-00a9cb4be95b\") " pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027025 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1cce4c8-a3ed-4133-b6bd-834c3b52f037-iptables-alerter-script\") pod \"iptables-alerter-b7wwv\" (UID: \"f1cce4c8-a3ed-4133-b6bd-834c3b52f037\") " pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:23.027095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027080 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-system-cni-dir\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027110 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-os-release\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-cnibin\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/caa17516-2b14-491b-aa49-c53308141da3-multus-daemon-config\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027183 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16baf6d7-7c56-4d29-82bd-36c91c330974-dbus\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027214 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027241 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caa17516-2b14-491b-aa49-c53308141da3-cni-binary-copy\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027253 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-cnibin\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1cce4c8-a3ed-4133-b6bd-834c3b52f037-host-slash\") pod \"iptables-alerter-b7wwv\" (UID: \"f1cce4c8-a3ed-4133-b6bd-834c3b52f037\") " pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsm7d\" (UniqueName: \"kubernetes.io/projected/c443b634-9b93-4578-b47b-00a9cb4be95b-kube-api-access-tsm7d\") pod \"node-ca-2vkng\" (UID: \"c443b634-9b93-4578-b47b-00a9cb4be95b\") " pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027306 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-os-release\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027324 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-cnibin\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-multus-socket-dir-parent\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027378 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-run-k8s-cni-cncf-io\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027406 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027406 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027434 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-run-netns\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.027848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027461 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-etc-kubernetes\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027485 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16baf6d7-7c56-4d29-82bd-36c91c330974-dbus\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027491 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6smth\" (UniqueName: \"kubernetes.io/projected/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-kube-api-access-6smth\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-var-lib-cni-bin\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027558 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-var-lib-kubelet\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027577 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1cce4c8-a3ed-4133-b6bd-834c3b52f037-iptables-alerter-script\") pod \"iptables-alerter-b7wwv\" (UID: \"f1cce4c8-a3ed-4133-b6bd-834c3b52f037\") " pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027593 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-run-multus-certs\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027646 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-multus-socket-dir-parent\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027660 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-run-multus-certs\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027716 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1cce4c8-a3ed-4133-b6bd-834c3b52f037-host-slash\") pod \"iptables-alerter-b7wwv\" (UID: \"f1cce4c8-a3ed-4133-b6bd-834c3b52f037\") " pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027718 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-var-lib-cni-bin\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027240 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027771 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-var-lib-kubelet\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027771 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-cnibin\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027307 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-hostroot\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.027858 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.027925 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret podName:16baf6d7-7c56-4d29-82bd-36c91c330974 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:23.527906065 +0000 UTC m=+2.085988642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret") pod "global-pull-secret-syncer-rq67m" (UID: "16baf6d7-7c56-4d29-82bd-36c91c330974") : object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:23.028494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027925 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/caa17516-2b14-491b-aa49-c53308141da3-cni-binary-copy\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.029082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027971 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/caa17516-2b14-491b-aa49-c53308141da3-multus-daemon-config\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.029082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.027989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-run-k8s-cni-cncf-io\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.029082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.028038 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-etc-kubernetes\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.029082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.028047 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/caa17516-2b14-491b-aa49-c53308141da3-host-run-netns\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.029082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.028197 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.032930 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.032911 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 03 16:04:23.032930 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.032933 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 03 16:04:23.033073 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.032943 2582 projected.go:196] Error preparing data for projected volume kube-api-access-4pkhn for pod openshift-network-diagnostics/network-check-target-p65bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:23.033073 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.032992 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn podName:36267cd5-1552-4c0b-ba41-a025192a0ea8 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:23.532978466 +0000 UTC m=+2.091061032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4pkhn" (UniqueName: "kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn") pod "network-check-target-p65bb" (UID: "36267cd5-1552-4c0b-ba41-a025192a0ea8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:23.035407 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.035381 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8xsn\" (UniqueName: \"kubernetes.io/projected/f1cce4c8-a3ed-4133-b6bd-834c3b52f037-kube-api-access-n8xsn\") pod \"iptables-alerter-b7wwv\" (UID: \"f1cce4c8-a3ed-4133-b6bd-834c3b52f037\") " pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:23.035890 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.035861 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cghp5\" (UniqueName: \"kubernetes.io/projected/caa17516-2b14-491b-aa49-c53308141da3-kube-api-access-cghp5\") pod \"multus-tqvvn\" (UID: \"caa17516-2b14-491b-aa49-c53308141da3\") " pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.038192 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.038151 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7f1ea143023130aaaa7a2dabe0813a.slice/crio-b951c096cdf5deec2f1d3e7f3a528068d07a142c0dd79122ede6f589fae221ba WatchSource:0}: Error finding container b951c096cdf5deec2f1d3e7f3a528068d07a142c0dd79122ede6f589fae221ba: Status 404 returned error can't find the container with id b951c096cdf5deec2f1d3e7f3a528068d07a142c0dd79122ede6f589fae221ba Aug 03 16:04:23.039040 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.039010 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsm7d\" (UniqueName: \"kubernetes.io/projected/c443b634-9b93-4578-b47b-00a9cb4be95b-kube-api-access-tsm7d\") pod \"node-ca-2vkng\" (UID: \"c443b634-9b93-4578-b47b-00a9cb4be95b\") " pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:23.039133 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.039071 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e5cbdaea4f5cac69ceab27cf785688.slice/crio-54a87837c3e4109f0dfc24f3c31ee4fc084be1e66bb55944e4714354dfc3a019 WatchSource:0}: Error finding container 54a87837c3e4109f0dfc24f3c31ee4fc084be1e66bb55944e4714354dfc3a019: Status 404 returned error can't find the container with id 54a87837c3e4109f0dfc24f3c31ee4fc084be1e66bb55944e4714354dfc3a019 Aug 03 16:04:23.039258 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.039238 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6smth\" (UniqueName: \"kubernetes.io/projected/c51fd627-9ce9-411d-ab28-8d3a7a8822e2-kube-api-access-6smth\") pod \"multus-additional-cni-plugins-lkckc\" (UID: \"c51fd627-9ce9-411d-ab28-8d3a7a8822e2\") " pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.043640 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.043626 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 03 16:04:23.139159 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.139067 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:23.141600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.141579 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 03 16:04:23.147913 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.147890 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a667877_9466_493b_beb0_fb6c194c11c9.slice/crio-b208656dcf25b701181c45658aee29148f93fbbdbef0192f969a0b252e5a138e WatchSource:0}: Error finding container b208656dcf25b701181c45658aee29148f93fbbdbef0192f969a0b252e5a138e: Status 404 returned error can't find the container with id b208656dcf25b701181c45658aee29148f93fbbdbef0192f969a0b252e5a138e Aug 03 16:04:23.154170 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.154152 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" Aug 03 16:04:23.161078 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.161052 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2150d1e5_3abc_47b1_a09a_37c8f04ab4ab.slice/crio-3b4208f6f88ff33cc04e35faf7e1bb002d0fc2bedbe63c9d8e3a3b200ee9c9ad WatchSource:0}: Error finding container 3b4208f6f88ff33cc04e35faf7e1bb002d0fc2bedbe63c9d8e3a3b200ee9c9ad: Status 404 returned error can't find the container with id 3b4208f6f88ff33cc04e35faf7e1bb002d0fc2bedbe63c9d8e3a3b200ee9c9ad Aug 03 16:04:23.174698 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.174680 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" Aug 03 16:04:23.184688 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.184657 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa6b0903_c5d7_4063_a62e_770d5ed6a211.slice/crio-cbf58766277fbc9a6c2db22bd62928e6b334c5cc7ef8b096687e3350b24c7edf WatchSource:0}: Error finding container cbf58766277fbc9a6c2db22bd62928e6b334c5cc7ef8b096687e3350b24c7edf: Status 404 returned error can't find the container with id cbf58766277fbc9a6c2db22bd62928e6b334c5cc7ef8b096687e3350b24c7edf Aug 03 16:04:23.188933 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.188910 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8g4rz" Aug 03 16:04:23.198111 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.198089 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a4751f_3a31_448f_a115_8d487b785137.slice/crio-cbdca1af27de29a24fe4a96917da3469a7ca30702485fd9d109800cfb173da00 WatchSource:0}: Error finding container cbdca1af27de29a24fe4a96917da3469a7ca30702485fd9d109800cfb173da00: Status 404 returned error can't find the container with id cbdca1af27de29a24fe4a96917da3469a7ca30702485fd9d109800cfb173da00 Aug 03 16:04:23.202846 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.202826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:23.210243 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.210222 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3fec6fc_6b35_45f2_ba0e_e0d57481b446.slice/crio-589c969ef414b72e66718db96eb9238d359ee4f30d9310229bef4a39ef381f2e WatchSource:0}: Error finding container 589c969ef414b72e66718db96eb9238d359ee4f30d9310229bef4a39ef381f2e: Status 404 returned error can't find the container with id 589c969ef414b72e66718db96eb9238d359ee4f30d9310229bef4a39ef381f2e Aug 03 16:04:23.211339 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.211321 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2vkng" Aug 03 16:04:23.218097 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.218078 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc443b634_9b93_4578_b47b_00a9cb4be95b.slice/crio-d6cc76adfddae95f7fb30c4dce3e8499523eb919beb849958bf179c5db18e418 WatchSource:0}: Error finding container d6cc76adfddae95f7fb30c4dce3e8499523eb919beb849958bf179c5db18e418: Status 404 returned error can't find the container with id d6cc76adfddae95f7fb30c4dce3e8499523eb919beb849958bf179c5db18e418 Aug 03 16:04:23.246144 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.246117 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lkckc" Aug 03 16:04:23.253329 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.253305 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51fd627_9ce9_411d_ab28_8d3a7a8822e2.slice/crio-1677660c42b15fb96d9b96d2b6b930a4bf7ae3c9e29a28b8ce023f914d36945c WatchSource:0}: Error finding container 1677660c42b15fb96d9b96d2b6b930a4bf7ae3c9e29a28b8ce023f914d36945c: Status 404 returned error can't find the container with id 1677660c42b15fb96d9b96d2b6b930a4bf7ae3c9e29a28b8ce023f914d36945c Aug 03 16:04:23.275777 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.275752 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tqvvn" Aug 03 16:04:23.280848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.280825 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b7wwv" Aug 03 16:04:23.282901 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.282878 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa17516_2b14_491b_aa49_c53308141da3.slice/crio-f884eb8511f70d69ea17dfebfd93994fc937899e9a2ec8ccb721209ce4559e85 WatchSource:0}: Error finding container f884eb8511f70d69ea17dfebfd93994fc937899e9a2ec8ccb721209ce4559e85: Status 404 returned error can't find the container with id f884eb8511f70d69ea17dfebfd93994fc937899e9a2ec8ccb721209ce4559e85 Aug 03 16:04:23.288035 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:04:23.288016 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1cce4c8_a3ed_4133_b6bd_834c3b52f037.slice/crio-da6a73102d08474b81459a523296790ffe238e962f21e5b87ca7c9f60fee8f5c WatchSource:0}: Error finding container da6a73102d08474b81459a523296790ffe238e962f21e5b87ca7c9f60fee8f5c: Status 404 returned error can't find the container with id da6a73102d08474b81459a523296790ffe238e962f21e5b87ca7c9f60fee8f5c Aug 03 16:04:23.297768 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.297748 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 03 16:04:23.430555 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.430450 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:23.430708 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.430601 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:23.430708 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.430676 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs podName:4baf391b-a641-4c8e-a1f0-e70a089ef564 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:24.430660258 +0000 UTC m=+2.988742822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs") pod "network-metrics-daemon-2dgwx" (UID: "4baf391b-a641-4c8e-a1f0-e70a089ef564") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:23.530892 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.530854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:23.531080 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.531041 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:23.531140 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.531112 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret podName:16baf6d7-7c56-4d29-82bd-36c91c330974 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:24.53109069 +0000 UTC m=+3.089173269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret") pod "global-pull-secret-syncer-rq67m" (UID: "16baf6d7-7c56-4d29-82bd-36c91c330974") : object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:23.631668 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.631630 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:23.631901 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.631809 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 03 16:04:23.631901 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.631833 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 03 16:04:23.631901 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.631846 2582 projected.go:196] Error preparing data for projected volume kube-api-access-4pkhn for pod openshift-network-diagnostics/network-check-target-p65bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:23.632073 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.631907 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn podName:36267cd5-1552-4c0b-ba41-a025192a0ea8 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:24.631887534 +0000 UTC m=+3.189970114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pkhn" (UniqueName: "kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn") pod "network-check-target-p65bb" (UID: "36267cd5-1552-4c0b-ba41-a025192a0ea8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:23.795720 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.795610 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 03 16:04:23.859718 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.857285 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 15:59:23 +0000 UTC" deadline="2028-01-12 17:51:27.927044841 +0000 UTC" Aug 03 16:04:23.859718 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.857352 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12649h47m4.069696813s" Aug 03 16:04:23.990831 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.989818 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:23.990831 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.989948 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:23.990831 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:23.990362 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:23.990831 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:23.990464 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:24.006822 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.006711 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2vkng" event={"ID":"c443b634-9b93-4578-b47b-00a9cb4be95b","Type":"ContainerStarted","Data":"d6cc76adfddae95f7fb30c4dce3e8499523eb919beb849958bf179c5db18e418"} Aug 03 16:04:24.027037 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.026908 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" event={"ID":"a3fec6fc-6b35-45f2-ba0e-e0d57481b446","Type":"ContainerStarted","Data":"589c969ef414b72e66718db96eb9238d359ee4f30d9310229bef4a39ef381f2e"} Aug 03 16:04:24.047757 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.047613 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8g4rz" event={"ID":"b9a4751f-3a31-448f-a115-8d487b785137","Type":"ContainerStarted","Data":"cbdca1af27de29a24fe4a96917da3469a7ca30702485fd9d109800cfb173da00"} Aug 03 16:04:24.071659 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.071621 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" event={"ID":"fa6b0903-c5d7-4063-a62e-770d5ed6a211","Type":"ContainerStarted","Data":"cbf58766277fbc9a6c2db22bd62928e6b334c5cc7ef8b096687e3350b24c7edf"} Aug 03 16:04:24.085367 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.085328 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" event={"ID":"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab","Type":"ContainerStarted","Data":"3b4208f6f88ff33cc04e35faf7e1bb002d0fc2bedbe63c9d8e3a3b200ee9c9ad"} Aug 03 16:04:24.099354 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.099315 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" event={"ID":"f1e5cbdaea4f5cac69ceab27cf785688","Type":"ContainerStarted","Data":"54a87837c3e4109f0dfc24f3c31ee4fc084be1e66bb55944e4714354dfc3a019"} Aug 03 16:04:24.112511 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.112473 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tqvvn" event={"ID":"caa17516-2b14-491b-aa49-c53308141da3","Type":"ContainerStarted","Data":"f884eb8511f70d69ea17dfebfd93994fc937899e9a2ec8ccb721209ce4559e85"} Aug 03 16:04:24.115724 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.115610 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n9756" event={"ID":"3a667877-9466-493b-beb0-fb6c194c11c9","Type":"ContainerStarted","Data":"b208656dcf25b701181c45658aee29148f93fbbdbef0192f969a0b252e5a138e"} Aug 03 16:04:24.126908 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.126875 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal" event={"ID":"6e7f1ea143023130aaaa7a2dabe0813a","Type":"ContainerStarted","Data":"b951c096cdf5deec2f1d3e7f3a528068d07a142c0dd79122ede6f589fae221ba"} Aug 03 16:04:24.145744 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.145706 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b7wwv" event={"ID":"f1cce4c8-a3ed-4133-b6bd-834c3b52f037","Type":"ContainerStarted","Data":"da6a73102d08474b81459a523296790ffe238e962f21e5b87ca7c9f60fee8f5c"} Aug 03 16:04:24.183319 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.183279 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkckc" event={"ID":"c51fd627-9ce9-411d-ab28-8d3a7a8822e2","Type":"ContainerStarted","Data":"1677660c42b15fb96d9b96d2b6b930a4bf7ae3c9e29a28b8ce023f914d36945c"} Aug 03 16:04:24.442767 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.442268 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:24.442767 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:24.442474 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:24.442767 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:24.442546 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs podName:4baf391b-a641-4c8e-a1f0-e70a089ef564 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:26.44252495 +0000 UTC m=+5.000607530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs") pod "network-metrics-daemon-2dgwx" (UID: "4baf391b-a641-4c8e-a1f0-e70a089ef564") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:24.543873 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.543221 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:24.543873 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:24.543402 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:24.543873 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:24.543474 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret podName:16baf6d7-7c56-4d29-82bd-36c91c330974 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:26.543454684 +0000 UTC m=+5.101537259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret") pod "global-pull-secret-syncer-rq67m" (UID: "16baf6d7-7c56-4d29-82bd-36c91c330974") : object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:24.644470 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.644431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:24.644663 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:24.644642 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 03 16:04:24.644765 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:24.644682 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 03 16:04:24.644765 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:24.644696 2582 projected.go:196] Error preparing data for projected volume kube-api-access-4pkhn for pod openshift-network-diagnostics/network-check-target-p65bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:24.644883 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:24.644798 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn podName:36267cd5-1552-4c0b-ba41-a025192a0ea8 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:26.644743749 +0000 UTC m=+5.202826319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pkhn" (UniqueName: "kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn") pod "network-check-target-p65bb" (UID: "36267cd5-1552-4c0b-ba41-a025192a0ea8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:24.858101 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.858007 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-02 15:59:23 +0000 UTC" deadline="2028-04-03 22:31:24.568386832 +0000 UTC" Aug 03 16:04:24.858101 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.858049 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14622h26m59.710342288s" Aug 03 16:04:24.989187 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:24.989153 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:24.989367 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:24.989299 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:25.989602 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:25.989398 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:25.989602 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:25.989553 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:25.990132 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:25.990118 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:25.990585 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:25.990244 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:26.460066 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:26.460019 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:26.460277 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:26.460185 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:26.460277 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:26.460251 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs podName:4baf391b-a641-4c8e-a1f0-e70a089ef564 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:30.46023197 +0000 UTC m=+9.018314541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs") pod "network-metrics-daemon-2dgwx" (UID: "4baf391b-a641-4c8e-a1f0-e70a089ef564") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:26.560793 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:26.560750 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:26.560990 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:26.560895 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:26.560990 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:26.560963 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret podName:16baf6d7-7c56-4d29-82bd-36c91c330974 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:30.560944125 +0000 UTC m=+9.119026693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret") pod "global-pull-secret-syncer-rq67m" (UID: "16baf6d7-7c56-4d29-82bd-36c91c330974") : object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:26.661538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:26.661493 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:26.661746 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:26.661701 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 03 16:04:26.661746 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:26.661725 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 03 16:04:26.661746 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:26.661738 2582 projected.go:196] Error preparing data for projected volume kube-api-access-4pkhn for pod openshift-network-diagnostics/network-check-target-p65bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:26.661927 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:26.661802 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn podName:36267cd5-1552-4c0b-ba41-a025192a0ea8 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:30.661782116 +0000 UTC m=+9.219864687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pkhn" (UniqueName: "kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn") pod "network-check-target-p65bb" (UID: "36267cd5-1552-4c0b-ba41-a025192a0ea8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:26.989601 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:26.989111 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:26.989601 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:26.989250 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:27.990386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:27.989842 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:27.990386 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:27.990003 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:27.990386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:27.990484 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:27.990386 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:27.990561 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:28.989752 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:28.989368 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:28.989752 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:28.989515 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:29.989510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:29.989474 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:29.989995 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:29.989616 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:29.989995 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:29.989838 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:29.989995 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:29.989952 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:30.497587 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:30.497541 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:30.497793 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:30.497749 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:30.497868 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:30.497842 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs podName:4baf391b-a641-4c8e-a1f0-e70a089ef564 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.49782169 +0000 UTC m=+17.055904256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs") pod "network-metrics-daemon-2dgwx" (UID: "4baf391b-a641-4c8e-a1f0-e70a089ef564") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:30.599025 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:30.598966 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:30.599207 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:30.599192 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:30.599307 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:30.599294 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret podName:16baf6d7-7c56-4d29-82bd-36c91c330974 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.599241159 +0000 UTC m=+17.157323740 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret") pod "global-pull-secret-syncer-rq67m" (UID: "16baf6d7-7c56-4d29-82bd-36c91c330974") : object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:30.700069 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:30.699809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:30.700069 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:30.700001 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 03 16:04:30.700069 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:30.700024 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 03 16:04:30.700069 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:30.700038 2582 projected.go:196] Error preparing data for projected volume kube-api-access-4pkhn for pod openshift-network-diagnostics/network-check-target-p65bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:30.700383 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:30.700102 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn podName:36267cd5-1552-4c0b-ba41-a025192a0ea8 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:38.700084066 +0000 UTC m=+17.258166642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pkhn" (UniqueName: "kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn") pod "network-check-target-p65bb" (UID: "36267cd5-1552-4c0b-ba41-a025192a0ea8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:30.989104 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:30.989015 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:30.989245 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:30.989143 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:31.989971 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:31.989935 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:31.990429 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:31.990074 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:31.990429 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:31.990291 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:31.990429 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:31.990407 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:32.989604 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:32.989521 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:32.989782 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:32.989690 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:33.989438 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:33.989398 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:33.989920 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:33.989441 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:33.989920 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:33.989563 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:33.989920 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:33.989691 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:34.989104 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:34.989029 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:34.989243 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:34.989133 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:35.988994 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:35.988954 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:35.989443 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:35.989104 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:35.989443 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:35.988954 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:35.989443 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:35.989258 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:36.989180 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:36.989146 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:36.989610 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:36.989274 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:37.989425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:37.989389 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:37.989834 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:37.989545 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:37.989834 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:37.989600 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:37.989834 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:37.989741 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:38.561768 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:38.561740 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:38.561875 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:38.561842 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:38.561913 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:38.561899 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs podName:4baf391b-a641-4c8e-a1f0-e70a089ef564 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:54.561882411 +0000 UTC m=+33.119964975 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs") pod "network-metrics-daemon-2dgwx" (UID: "4baf391b-a641-4c8e-a1f0-e70a089ef564") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:38.662578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:38.662504 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:38.662751 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:38.662613 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:38.662751 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:38.662704 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret podName:16baf6d7-7c56-4d29-82bd-36c91c330974 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:54.662684149 +0000 UTC m=+33.220766722 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret") pod "global-pull-secret-syncer-rq67m" (UID: "16baf6d7-7c56-4d29-82bd-36c91c330974") : object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:38.763708 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:38.763661 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:38.763884 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:38.763850 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 03 16:04:38.763884 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:38.763881 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 03 16:04:38.763960 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:38.763895 2582 projected.go:196] Error preparing data for projected volume kube-api-access-4pkhn for pod openshift-network-diagnostics/network-check-target-p65bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:38.764001 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:38.763961 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn podName:36267cd5-1552-4c0b-ba41-a025192a0ea8 nodeName:}" failed. No retries permitted until 2026-08-03 16:04:54.763943674 +0000 UTC m=+33.322026254 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pkhn" (UniqueName: "kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn") pod "network-check-target-p65bb" (UID: "36267cd5-1552-4c0b-ba41-a025192a0ea8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:38.989117 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:38.989036 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:38.989267 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:38.989137 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:39.989418 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:39.989378 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:39.989916 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:39.989433 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:39.989916 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:39.989532 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:39.989916 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:39.989611 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:40.989204 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:40.989168 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:40.989381 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:40.989307 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:41.989543 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:41.989500 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:41.989979 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:41.989627 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:41.989979 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:41.989676 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:41.989979 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:41.989768 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:42.989561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:42.989383 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:42.990065 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:42.989768 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:43.238166 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.237931 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tqvvn" event={"ID":"caa17516-2b14-491b-aa49-c53308141da3","Type":"ContainerStarted","Data":"7ab69664c44a23ba581d5e054bdb61e0e5aeb857238e0591157dbaefaa5529a2"} Aug 03 16:04:43.239586 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.239555 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal" event={"ID":"6e7f1ea143023130aaaa7a2dabe0813a","Type":"ContainerStarted","Data":"b73ca246b1a08e8aa9677d955f44059637b80e11d00947bf5328ef935fb6b78d"} Aug 03 16:04:43.242405 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.242373 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" event={"ID":"a3fec6fc-6b35-45f2-ba0e-e0d57481b446","Type":"ContainerStarted","Data":"5413a7b8f7a4552d124c7007c2cb212d0c0e2a7f8106edacd85bd0db973f7897"} Aug 03 16:04:43.242512 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.242432 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" event={"ID":"a3fec6fc-6b35-45f2-ba0e-e0d57481b446","Type":"ContainerStarted","Data":"3a4ce64027020c086647ac3fdffc28ba3fad2a1d7b7970a0ea6192bfbbbcdb8a"} Aug 03 16:04:43.243718 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.243695 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" event={"ID":"fa6b0903-c5d7-4063-a62e-770d5ed6a211","Type":"ContainerStarted","Data":"2e2d6c69d1b512ebc609b999d729befd05e779ff85a33add5c3408c7420cd3e5"} Aug 03 16:04:43.256980 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.256940 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tqvvn" podStartSLOduration=2.053196503 podStartE2EDuration="21.256926681s" totalImagesPullingTime="19.203730178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:43.256377456 +0000 UTC m=+21.814460040" watchObservedRunningTime="2026-08-03 16:04:43.256926681 +0000 UTC m=+21.815009264" Aug 03 16:04:43.270360 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.270330 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-140.ec2.internal" podStartSLOduration=20.270318089 podStartE2EDuration="20.270318089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:43.270110444 +0000 UTC m=+21.828193031" watchObservedRunningTime="2026-08-03 16:04:43.270318089 +0000 UTC m=+21.828400673" Aug 03 16:04:43.287441 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.287405 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6xxtl" podStartSLOduration=1.984721172 podStartE2EDuration="21.287391961s" totalImagesPullingTime="19.302670789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:43.287359696 +0000 UTC m=+21.845442275" watchObservedRunningTime="2026-08-03 16:04:43.287391961 +0000 UTC m=+21.845474536" Aug 03 16:04:43.989551 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.989515 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:43.989745 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:43.989519 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:43.989745 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:43.989697 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:43.990653 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:43.989757 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:44.248057 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.247970 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b7wwv" event={"ID":"f1cce4c8-a3ed-4133-b6bd-834c3b52f037","Type":"ContainerStarted","Data":"777aa52f8bc7d6f004f7388a290f360cc664e306b5d7130a31780d0855462223"} Aug 03 16:04:44.249903 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.249864 2582 generic.go:356] "Generic (PLEG): container finished" podID="c51fd627-9ce9-411d-ab28-8d3a7a8822e2" containerID="23ac7f34930de6d1aad692225d3c45e00e8dbc3a4c4b3435a3ae657070ab05b2" exitCode=0 Aug 03 16:04:44.250020 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.249968 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkckc" event={"ID":"c51fd627-9ce9-411d-ab28-8d3a7a8822e2","Type":"ContainerDied","Data":"23ac7f34930de6d1aad692225d3c45e00e8dbc3a4c4b3435a3ae657070ab05b2"} Aug 03 16:04:44.251456 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.251437 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2vkng" event={"ID":"c443b634-9b93-4578-b47b-00a9cb4be95b","Type":"ContainerStarted","Data":"e485d6a88ae59eae2b4cdbf9f656683fd7e90439d57e3a8e65bb40c4de55deeb"} Aug 03 16:04:44.254581 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.254562 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" event={"ID":"a3fec6fc-6b35-45f2-ba0e-e0d57481b446","Type":"ContainerStarted","Data":"2ad4579e7993f7458548e65d7e4d35816c9dfb44fadba51a5c7348d83f69987e"} Aug 03 16:04:44.254677 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.254629 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" event={"ID":"a3fec6fc-6b35-45f2-ba0e-e0d57481b446","Type":"ContainerStarted","Data":"21d363e5c14a159b142a3710c5086c2adb32bbf4a2d10ce779668057d44bd6b2"} Aug 03 16:04:44.254677 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.254661 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" event={"ID":"a3fec6fc-6b35-45f2-ba0e-e0d57481b446","Type":"ContainerStarted","Data":"1947a77c0651437f1d37d0f2fe09a886d8d9a637b01aa111f26afc90fd24cfe8"} Aug 03 16:04:44.254796 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.254744 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" event={"ID":"a3fec6fc-6b35-45f2-ba0e-e0d57481b446","Type":"ContainerStarted","Data":"62c2331884ad970d53564b00aafd9ecf5eb0c609f17b878bdc6bc4f13f38fdf4"} Aug 03 16:04:44.258104 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.258081 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8g4rz" event={"ID":"b9a4751f-3a31-448f-a115-8d487b785137","Type":"ContainerStarted","Data":"c4c69982787d35747d9f157a2826208742037924779b86841577f1162a50be4f"} Aug 03 16:04:44.259211 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.259195 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" event={"ID":"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab","Type":"ContainerStarted","Data":"7500ba75b60ce8102cfe17e0dd1e09b7eeeb8c68b747fbbe941d1aea8d08afae"} Aug 03 16:04:44.260454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.260436 2582 generic.go:356] "Generic (PLEG): container finished" podID="f1e5cbdaea4f5cac69ceab27cf785688" containerID="277e61834c05fc19b09ed102163bb61513978926a90c4922a118869674d91196" exitCode=0 Aug 03 16:04:44.260541 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.260492 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" event={"ID":"f1e5cbdaea4f5cac69ceab27cf785688","Type":"ContainerDied","Data":"277e61834c05fc19b09ed102163bb61513978926a90c4922a118869674d91196"} Aug 03 16:04:44.261709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.261687 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n9756" event={"ID":"3a667877-9466-493b-beb0-fb6c194c11c9","Type":"ContainerStarted","Data":"8fb765293394e8b86807b254287a3e3e484f92c46a9feb1a7b86678969a399d3"} Aug 03 16:04:44.263699 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.263637 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b7wwv" podStartSLOduration=3.060234719 podStartE2EDuration="22.263612121s" totalImagesPullingTime="19.203377402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:44.263488338 +0000 UTC m=+22.821570922" watchObservedRunningTime="2026-08-03 16:04:44.263612121 +0000 UTC m=+22.821694705" Aug 03 16:04:44.278347 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.278311 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8g4rz" podStartSLOduration=2.989618409 podStartE2EDuration="22.278299382s" totalImagesPullingTime="19.288680973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:44.278244652 +0000 UTC m=+22.836327236" watchObservedRunningTime="2026-08-03 16:04:44.278299382 +0000 UTC m=+22.836381960" Aug 03 16:04:44.292942 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.292908 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2vkng" podStartSLOduration=6.927194076 podStartE2EDuration="22.29289829s" totalImagesPullingTime="15.365704214s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:44.292459694 +0000 UTC m=+22.850542280" watchObservedRunningTime="2026-08-03 16:04:44.29289829 +0000 UTC m=+22.850980875" Aug 03 16:04:44.331823 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.331488 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n9756" podStartSLOduration=2.993661529 podStartE2EDuration="22.331472063s" totalImagesPullingTime="19.337810534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:44.306855249 +0000 UTC m=+22.864937834" watchObservedRunningTime="2026-08-03 16:04:44.331472063 +0000 UTC m=+22.889554648" Aug 03 16:04:44.597790 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.597764 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 03 16:04:44.879170 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.879006 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-03T16:04:44.597785119Z","UUID":"13208dfd-3e23-4c9f-8c83-dc6faa993eeb","Handler":null,"Name":"","Endpoint":""} Aug 03 16:04:44.881123 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.881103 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 03 16:04:44.881256 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.881144 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 03 16:04:44.989723 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:44.989692 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:44.989967 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:44.989821 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:45.266693 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:45.266597 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" event={"ID":"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab","Type":"ContainerStarted","Data":"41e90b4b179f827431c302b700758d2375748f1ee63de0429709a3e1e78d66fb"} Aug 03 16:04:45.268664 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:45.268635 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" event={"ID":"f1e5cbdaea4f5cac69ceab27cf785688","Type":"ContainerStarted","Data":"e6e5b5a162091a4077a8f01b8e42fe4c1a0868df21d23d6ab21991c4eabd8bdb"} Aug 03 16:04:45.283510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:45.283462 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-140.ec2.internal" podStartSLOduration=22.283445579 podStartE2EDuration="22.283445579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:45.283323741 +0000 UTC m=+23.841406329" watchObservedRunningTime="2026-08-03 16:04:45.283445579 +0000 UTC m=+23.841528169" Aug 03 16:04:45.989721 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:45.989687 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:45.989721 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:45.989720 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:45.989954 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:45.989836 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:45.990290 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:45.990250 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:46.274849 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:46.274765 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" event={"ID":"a3fec6fc-6b35-45f2-ba0e-e0d57481b446","Type":"ContainerStarted","Data":"5805985963cd3c799f9009d9e690d0625e7dfc6421680808b7120a6a12726106"} Aug 03 16:04:46.277091 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:46.277064 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" event={"ID":"2150d1e5-3abc-47b1-a09a-37c8f04ab4ab","Type":"ContainerStarted","Data":"53a958383d4cb0c88b1048a9ef1311f13d60aea3a8df86f051365346f05d1072"} Aug 03 16:04:46.297413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:46.297360 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lnnnq" podStartSLOduration=2.823930586 podStartE2EDuration="24.297342181s" totalImagesPullingTime="21.473411595s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:46.296014286 +0000 UTC m=+24.854096865" watchObservedRunningTime="2026-08-03 16:04:46.297342181 +0000 UTC m=+24.855424768" Aug 03 16:04:46.989497 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:46.989464 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:46.989809 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:46.989586 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:47.651946 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:47.651905 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:47.652541 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:47.652524 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:47.989532 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:47.989491 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:47.989739 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:47.989612 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:47.989739 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:47.989675 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:47.989867 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:47.989823 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:48.280912 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:48.280882 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:48.281353 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:48.281338 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n9756" Aug 03 16:04:48.989865 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:48.989653 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:48.990324 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:48.989899 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:49.284393 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.284317 2582 generic.go:356] "Generic (PLEG): container finished" podID="c51fd627-9ce9-411d-ab28-8d3a7a8822e2" containerID="b514d65e3c6c1f534ba19888c05d1d2d5124cff1b077520b4348081a99d02ac3" exitCode=0 Aug 03 16:04:49.284543 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.284396 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkckc" event={"ID":"c51fd627-9ce9-411d-ab28-8d3a7a8822e2","Type":"ContainerDied","Data":"b514d65e3c6c1f534ba19888c05d1d2d5124cff1b077520b4348081a99d02ac3"} Aug 03 16:04:49.287805 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.287783 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" event={"ID":"a3fec6fc-6b35-45f2-ba0e-e0d57481b446","Type":"ContainerStarted","Data":"1a50a4401841b45080305a1e9afafd29b38ef768a0f26a0925eb8cb6f63e900e"} Aug 03 16:04:49.288022 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.287997 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:49.288145 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.288069 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:49.288145 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.288120 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:49.302575 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.302545 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:49.302736 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.302706 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:04:49.335154 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.335117 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" podStartSLOduration=7.596618004 podStartE2EDuration="27.33510394s" totalImagesPullingTime="19.738485936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:04:49.33486105 +0000 UTC m=+27.892943635" watchObservedRunningTime="2026-08-03 16:04:49.33510394 +0000 UTC m=+27.893186524" Aug 03 16:04:49.989191 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.989158 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:49.989401 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:49.989259 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:49.989401 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:49.989318 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:49.989513 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:49.989439 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:50.268416 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:50.268186 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2dgwx"] Aug 03 16:04:50.268903 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:50.268691 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p65bb"] Aug 03 16:04:50.283408 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:50.283386 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rq67m"] Aug 03 16:04:50.283533 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:50.283502 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:50.283669 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:50.283608 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:50.289452 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:50.289433 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:50.289565 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:50.289531 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:50.289565 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:50.289555 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:50.289682 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:50.289636 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:50.754918 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:50.754887 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8g4rz_b9a4751f-3a31-448f-a115-8d487b785137/dns-node-resolver/0.log" Aug 03 16:04:51.526727 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:51.526656 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2vkng_c443b634-9b93-4578-b47b-00a9cb4be95b/node-ca/0.log" Aug 03 16:04:51.989909 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:51.989826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:51.990069 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:51.989924 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:51.990069 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:51.989958 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:51.990069 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:51.989997 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:51.990069 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:51.990023 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:51.990259 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:51.990105 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:52.296576 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:52.296505 2582 generic.go:356] "Generic (PLEG): container finished" podID="c51fd627-9ce9-411d-ab28-8d3a7a8822e2" containerID="a2c17ec396af0d760bee70a1034b953d9c653bdb83330bf71adcc126abd35467" exitCode=0 Aug 03 16:04:52.296576 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:52.296561 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkckc" event={"ID":"c51fd627-9ce9-411d-ab28-8d3a7a8822e2","Type":"ContainerDied","Data":"a2c17ec396af0d760bee70a1034b953d9c653bdb83330bf71adcc126abd35467"} Aug 03 16:04:53.300843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:53.300719 2582 generic.go:356] "Generic (PLEG): container finished" podID="c51fd627-9ce9-411d-ab28-8d3a7a8822e2" containerID="de5d2f7f9ea1080f136defda452f4f660c4b8dcadae08b4aa380c0453f53226b" exitCode=0 Aug 03 16:04:53.300843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:53.300757 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkckc" event={"ID":"c51fd627-9ce9-411d-ab28-8d3a7a8822e2","Type":"ContainerDied","Data":"de5d2f7f9ea1080f136defda452f4f660c4b8dcadae08b4aa380c0453f53226b"} Aug 03 16:04:53.992153 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:53.992124 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:53.992290 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:53.992216 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:53.992290 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:53.992224 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:53.992290 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:53.992240 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:53.992403 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:53.992315 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:53.992403 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:53.992375 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:54.595703 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:54.595664 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:54.596098 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:54.595812 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:54.596098 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:54.595877 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs podName:4baf391b-a641-4c8e-a1f0-e70a089ef564 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:26.595860527 +0000 UTC m=+65.153943093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs") pod "network-metrics-daemon-2dgwx" (UID: "4baf391b-a641-4c8e-a1f0-e70a089ef564") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 03 16:04:54.696305 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:54.696273 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:54.696472 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:54.696417 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:54.696511 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:54.696481 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret podName:16baf6d7-7c56-4d29-82bd-36c91c330974 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:26.696467342 +0000 UTC m=+65.254549905 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret") pod "global-pull-secret-syncer-rq67m" (UID: "16baf6d7-7c56-4d29-82bd-36c91c330974") : object "kube-system"/"original-pull-secret" not registered Aug 03 16:04:54.797476 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:54.797441 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:54.797678 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:54.797585 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 03 16:04:54.797678 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:54.797604 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 03 16:04:54.797678 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:54.797627 2582 projected.go:196] Error preparing data for projected volume kube-api-access-4pkhn for pod openshift-network-diagnostics/network-check-target-p65bb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:54.797812 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:54.797685 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn podName:36267cd5-1552-4c0b-ba41-a025192a0ea8 nodeName:}" failed. No retries permitted until 2026-08-03 16:05:26.797672131 +0000 UTC m=+65.355754698 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4pkhn" (UniqueName: "kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn") pod "network-check-target-p65bb" (UID: "36267cd5-1552-4c0b-ba41-a025192a0ea8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 03 16:04:55.989484 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:55.989433 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:55.989968 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:55.989433 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:55.989968 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:55.989558 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:55.989968 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:55.989604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:55.989968 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:55.989743 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:55.989968 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:55.989821 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:57.992957 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:57.992928 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:57.993398 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:57.993038 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:57.993398 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:57.993045 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:57.993398 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:57.993057 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:57.993398 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:57.993149 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:04:57.993398 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:57.993250 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:59.316790 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:59.316612 2582 generic.go:356] "Generic (PLEG): container finished" podID="c51fd627-9ce9-411d-ab28-8d3a7a8822e2" containerID="ac2dd22bf2f11de3aced37e096c309628f95b73ff225b8eb86417484acd0226f" exitCode=0 Aug 03 16:04:59.316790 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:59.316655 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkckc" event={"ID":"c51fd627-9ce9-411d-ab28-8d3a7a8822e2","Type":"ContainerDied","Data":"ac2dd22bf2f11de3aced37e096c309628f95b73ff225b8eb86417484acd0226f"} Aug 03 16:04:59.989038 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:59.989002 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:04:59.989224 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:59.989002 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:04:59.989224 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:59.989128 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:04:59.989224 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:04:59.989011 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:04:59.989224 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:59.989174 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:04:59.989442 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:04:59.989281 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:05:00.321401 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:00.321320 2582 generic.go:356] "Generic (PLEG): container finished" podID="c51fd627-9ce9-411d-ab28-8d3a7a8822e2" containerID="483e9452cc6fa8a13cac22ae205f4e658cc9b33c083dccbf52fef636e01aaa80" exitCode=0 Aug 03 16:05:00.321401 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:00.321373 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkckc" event={"ID":"c51fd627-9ce9-411d-ab28-8d3a7a8822e2","Type":"ContainerDied","Data":"483e9452cc6fa8a13cac22ae205f4e658cc9b33c083dccbf52fef636e01aaa80"} Aug 03 16:05:01.326707 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:01.326676 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkckc" event={"ID":"c51fd627-9ce9-411d-ab28-8d3a7a8822e2","Type":"ContainerStarted","Data":"ab18825a627ba6bfcd02fbcd6a5eb1735a91f8b1e28f1860ba4709193b9badfa"} Aug 03 16:05:01.365452 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:01.365408 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lkckc" podStartSLOduration=7.860175832 podStartE2EDuration="39.365393168s" totalImagesPullingTime="31.505217336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:01.365208832 +0000 UTC m=+39.923291416" watchObservedRunningTime="2026-08-03 16:05:01.365393168 +0000 UTC m=+39.923475753" Aug 03 16:05:01.990116 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:01.990082 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:01.990279 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:01.990183 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:01.990279 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:01.990223 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:01.990279 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:01.990218 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:05:01.990378 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:01.990307 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:05:01.990411 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:01.990386 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:05:03.989578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:03.989542 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:03.989949 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:03.989676 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:03.989949 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:03.989783 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:05:03.989949 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:03.989680 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:05:03.989949 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:03.989826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:03.989949 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:03.989906 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:05:05.989523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:05.989478 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:05.990102 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:05.989497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:05.990102 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:05.989609 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:05:05.990102 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:05.989734 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:05.990102 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:05.989729 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:05:05.990102 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:05.989791 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:05:07.989700 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:07.989665 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:07.989700 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:07.989694 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:07.990137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:07.989664 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:07.990137 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:07.989777 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:05:07.990137 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:07.989838 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:05:07.990137 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:07.989891 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:05:09.989141 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:09.989114 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:09.989141 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:09.989150 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:09.989664 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:09.989119 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:09.989664 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:09.989244 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:05:09.989664 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:09.989319 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:05:09.989664 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:09.989386 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:05:11.990404 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:11.990374 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:11.990781 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:11.990442 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:11.990781 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:11.990450 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p65bb" podUID="36267cd5-1552-4c0b-ba41-a025192a0ea8" Aug 03 16:05:11.990781 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:11.990517 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rq67m" podUID="16baf6d7-7c56-4d29-82bd-36c91c330974" Aug 03 16:05:11.990781 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:11.990560 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:11.990781 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:11.990627 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2dgwx" podUID="4baf391b-a641-4c8e-a1f0-e70a089ef564" Aug 03 16:05:13.807041 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.807016 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-140.ec2.internal" event="NodeReady" Aug 03 16:05:13.807458 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.807113 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 03 16:05:13.848692 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.848662 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7hj45"] Aug 03 16:05:13.850368 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.850355 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7hj45" Aug 03 16:05:13.853259 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.853239 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 03 16:05:13.853259 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.853239 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 03 16:05:13.853417 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.853267 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dkjck\"" Aug 03 16:05:13.853417 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.853287 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 03 16:05:13.857699 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.856311 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4xtzr"] Aug 03 16:05:13.858780 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.858764 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:13.860977 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.860956 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7hj45"] Aug 03 16:05:13.861443 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.861422 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 03 16:05:13.861588 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.861524 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-grgrw\"" Aug 03 16:05:13.861730 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.861691 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 03 16:05:13.861843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.861770 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 03 16:05:13.861843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.861834 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 03 16:05:13.868700 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.868681 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4xtzr"] Aug 03 16:05:13.871015 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.870994 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xwhgl"] Aug 03 16:05:13.872903 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.872886 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:13.875555 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.875537 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vz785\"" Aug 03 16:05:13.875646 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.875557 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 03 16:05:13.875646 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.875577 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 03 16:05:13.883053 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.883033 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xwhgl"] Aug 03 16:05:13.935964 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.935935 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc59383f-9acf-44f5-85a0-fe7eb7189356-tmp-dir\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:13.936128 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.935974 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b4167a63-fa6c-427a-8be0-51fb4be4540d-crio-socket\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:13.936128 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.936007 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1024e841-024a-4af9-a182-c08ab00e2bf8-cert\") pod \"ingress-canary-7hj45\" (UID: \"1024e841-024a-4af9-a182-c08ab00e2bf8\") " pod="openshift-ingress-canary/ingress-canary-7hj45" Aug 03 16:05:13.936128 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.936062 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptm8l\" (UniqueName: \"kubernetes.io/projected/b4167a63-fa6c-427a-8be0-51fb4be4540d-kube-api-access-ptm8l\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:13.936128 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.936096 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v85rr\" (UniqueName: \"kubernetes.io/projected/bc59383f-9acf-44f5-85a0-fe7eb7189356-kube-api-access-v85rr\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:13.936128 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.936127 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4167a63-fa6c-427a-8be0-51fb4be4540d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:13.936301 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.936165 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc59383f-9acf-44f5-85a0-fe7eb7189356-config-volume\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:13.936301 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.936190 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc59383f-9acf-44f5-85a0-fe7eb7189356-metrics-tls\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:13.936301 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.936204 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4167a63-fa6c-427a-8be0-51fb4be4540d-data-volume\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:13.936301 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.936218 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4167a63-fa6c-427a-8be0-51fb4be4540d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:13.936301 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.936233 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4v2r\" (UniqueName: \"kubernetes.io/projected/1024e841-024a-4af9-a182-c08ab00e2bf8-kube-api-access-f4v2r\") pod \"ingress-canary-7hj45\" (UID: \"1024e841-024a-4af9-a182-c08ab00e2bf8\") " pod="openshift-ingress-canary/ingress-canary-7hj45" Aug 03 16:05:13.989090 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.989056 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:13.989239 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.989056 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:13.989323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.989056 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:13.992309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.992276 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 03 16:05:13.992309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.992306 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 03 16:05:13.992494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.992365 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xrdkc\"" Aug 03 16:05:13.992494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.992367 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzpf\"" Aug 03 16:05:13.992494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.992400 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 03 16:05:13.992643 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:13.992510 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 03 16:05:14.036843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.036816 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b4167a63-fa6c-427a-8be0-51fb4be4540d-crio-socket\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.036995 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.036851 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1024e841-024a-4af9-a182-c08ab00e2bf8-cert\") pod \"ingress-canary-7hj45\" (UID: \"1024e841-024a-4af9-a182-c08ab00e2bf8\") " pod="openshift-ingress-canary/ingress-canary-7hj45" Aug 03 16:05:14.036995 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.036873 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptm8l\" (UniqueName: \"kubernetes.io/projected/b4167a63-fa6c-427a-8be0-51fb4be4540d-kube-api-access-ptm8l\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.036995 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.036894 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v85rr\" (UniqueName: \"kubernetes.io/projected/bc59383f-9acf-44f5-85a0-fe7eb7189356-kube-api-access-v85rr\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:14.036995 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.036925 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4167a63-fa6c-427a-8be0-51fb4be4540d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.036995 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.036944 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b4167a63-fa6c-427a-8be0-51fb4be4540d-crio-socket\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.037239 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037059 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc59383f-9acf-44f5-85a0-fe7eb7189356-config-volume\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:14.037239 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037089 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc59383f-9acf-44f5-85a0-fe7eb7189356-metrics-tls\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:14.037239 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037113 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4167a63-fa6c-427a-8be0-51fb4be4540d-data-volume\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.037239 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037139 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4167a63-fa6c-427a-8be0-51fb4be4540d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.037239 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037164 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4v2r\" (UniqueName: \"kubernetes.io/projected/1024e841-024a-4af9-a182-c08ab00e2bf8-kube-api-access-f4v2r\") pod \"ingress-canary-7hj45\" (UID: \"1024e841-024a-4af9-a182-c08ab00e2bf8\") " pod="openshift-ingress-canary/ingress-canary-7hj45" Aug 03 16:05:14.037239 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037220 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc59383f-9acf-44f5-85a0-fe7eb7189356-tmp-dir\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:14.037498 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037414 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4167a63-fa6c-427a-8be0-51fb4be4540d-data-volume\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.037556 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037542 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc59383f-9acf-44f5-85a0-fe7eb7189356-tmp-dir\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:14.037699 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037681 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bc59383f-9acf-44f5-85a0-fe7eb7189356-config-volume\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:14.037741 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.037717 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4167a63-fa6c-427a-8be0-51fb4be4540d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.040981 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.040955 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4167a63-fa6c-427a-8be0-51fb4be4540d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.041067 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.040986 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bc59383f-9acf-44f5-85a0-fe7eb7189356-metrics-tls\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:14.041113 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.041069 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1024e841-024a-4af9-a182-c08ab00e2bf8-cert\") pod \"ingress-canary-7hj45\" (UID: \"1024e841-024a-4af9-a182-c08ab00e2bf8\") " pod="openshift-ingress-canary/ingress-canary-7hj45" Aug 03 16:05:14.045026 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.045004 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v85rr\" (UniqueName: \"kubernetes.io/projected/bc59383f-9acf-44f5-85a0-fe7eb7189356-kube-api-access-v85rr\") pod \"dns-default-xwhgl\" (UID: \"bc59383f-9acf-44f5-85a0-fe7eb7189356\") " pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:14.045539 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.045518 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4v2r\" (UniqueName: \"kubernetes.io/projected/1024e841-024a-4af9-a182-c08ab00e2bf8-kube-api-access-f4v2r\") pod \"ingress-canary-7hj45\" (UID: \"1024e841-024a-4af9-a182-c08ab00e2bf8\") " pod="openshift-ingress-canary/ingress-canary-7hj45" Aug 03 16:05:14.045690 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.045670 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptm8l\" (UniqueName: \"kubernetes.io/projected/b4167a63-fa6c-427a-8be0-51fb4be4540d-kube-api-access-ptm8l\") pod \"insights-runtime-extractor-4xtzr\" (UID: \"b4167a63-fa6c-427a-8be0-51fb4be4540d\") " pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.162524 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.162440 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7hj45" Aug 03 16:05:14.170066 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.170047 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4xtzr" Aug 03 16:05:14.180555 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.180531 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:14.340725 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.340695 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7hj45"] Aug 03 16:05:14.343357 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:14.343335 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1024e841_024a_4af9_a182_c08ab00e2bf8.slice/crio-f32ef47f196543303b757fa15de402636b2d77e28fe82ca6f31a8813e73cceae WatchSource:0}: Error finding container f32ef47f196543303b757fa15de402636b2d77e28fe82ca6f31a8813e73cceae: Status 404 returned error can't find the container with id f32ef47f196543303b757fa15de402636b2d77e28fe82ca6f31a8813e73cceae Aug 03 16:05:14.354983 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.354957 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7hj45" event={"ID":"1024e841-024a-4af9-a182-c08ab00e2bf8","Type":"ContainerStarted","Data":"f32ef47f196543303b757fa15de402636b2d77e28fe82ca6f31a8813e73cceae"} Aug 03 16:05:14.530992 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.530958 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4xtzr"] Aug 03 16:05:14.531907 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.531726 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xwhgl"] Aug 03 16:05:14.533795 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:14.533766 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc59383f_9acf_44f5_85a0_fe7eb7189356.slice/crio-b6eee763796ce223b850a97178628ba7848f852a80ff557f20c859c70c72260f WatchSource:0}: Error finding container b6eee763796ce223b850a97178628ba7848f852a80ff557f20c859c70c72260f: Status 404 returned error can't find the container with id b6eee763796ce223b850a97178628ba7848f852a80ff557f20c859c70c72260f Aug 03 16:05:14.534553 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:14.534531 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4167a63_fa6c_427a_8be0_51fb4be4540d.slice/crio-1f7f222370c9806cb022c32377446803a9c708b556ad249b4bacace3cf0a0faa WatchSource:0}: Error finding container 1f7f222370c9806cb022c32377446803a9c708b556ad249b4bacace3cf0a0faa: Status 404 returned error can't find the container with id 1f7f222370c9806cb022c32377446803a9c708b556ad249b4bacace3cf0a0faa Aug 03 16:05:14.779243 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.779214 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99"] Aug 03 16:05:14.781402 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.781351 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" Aug 03 16:05:14.783926 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.783907 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Aug 03 16:05:14.784018 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.783979 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ggnjh\"" Aug 03 16:05:14.790287 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.790265 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99"] Aug 03 16:05:14.844879 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.844844 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6f748fd-5da6-4f68-9786-0975f6fabdbb-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-z4w99\" (UID: \"d6f748fd-5da6-4f68-9786-0975f6fabdbb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" Aug 03 16:05:14.945924 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:14.945890 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6f748fd-5da6-4f68-9786-0975f6fabdbb-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-z4w99\" (UID: \"d6f748fd-5da6-4f68-9786-0975f6fabdbb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" Aug 03 16:05:14.946156 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:14.946054 2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Aug 03 16:05:14.946156 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:14.946131 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6f748fd-5da6-4f68-9786-0975f6fabdbb-tls-certificates podName:d6f748fd-5da6-4f68-9786-0975f6fabdbb nodeName:}" failed. No retries permitted until 2026-08-03 16:05:15.446110968 +0000 UTC m=+54.004193554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/d6f748fd-5da6-4f68-9786-0975f6fabdbb-tls-certificates") pod "prometheus-operator-admission-webhook-66685ff6d4-z4w99" (UID: "d6f748fd-5da6-4f68-9786-0975f6fabdbb") : secret "prometheus-operator-admission-webhook-tls" not found Aug 03 16:05:15.359023 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:15.358984 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xwhgl" event={"ID":"bc59383f-9acf-44f5-85a0-fe7eb7189356","Type":"ContainerStarted","Data":"b6eee763796ce223b850a97178628ba7848f852a80ff557f20c859c70c72260f"} Aug 03 16:05:15.361047 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:15.361019 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4xtzr" event={"ID":"b4167a63-fa6c-427a-8be0-51fb4be4540d","Type":"ContainerStarted","Data":"c1d277b155c78053e2e3e5d92084ae7c49e09ab28651ffab2492c71d2947b3d0"} Aug 03 16:05:15.361166 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:15.361075 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4xtzr" event={"ID":"b4167a63-fa6c-427a-8be0-51fb4be4540d","Type":"ContainerStarted","Data":"cfa8b5885bef204d922a1d64044adf3721d5888dc6ca79f1bc1fbfd2488632c9"} Aug 03 16:05:15.361166 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:15.361102 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4xtzr" event={"ID":"b4167a63-fa6c-427a-8be0-51fb4be4540d","Type":"ContainerStarted","Data":"1f7f222370c9806cb022c32377446803a9c708b556ad249b4bacace3cf0a0faa"} Aug 03 16:05:15.450471 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:15.450379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6f748fd-5da6-4f68-9786-0975f6fabdbb-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-z4w99\" (UID: \"d6f748fd-5da6-4f68-9786-0975f6fabdbb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" Aug 03 16:05:15.453356 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:15.453327 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d6f748fd-5da6-4f68-9786-0975f6fabdbb-tls-certificates\") pod \"prometheus-operator-admission-webhook-66685ff6d4-z4w99\" (UID: \"d6f748fd-5da6-4f68-9786-0975f6fabdbb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" Aug 03 16:05:15.689340 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:15.689301 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" Aug 03 16:05:16.230743 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:16.230714 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99"] Aug 03 16:05:16.883893 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:16.883861 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f748fd_5da6_4f68_9786_0975f6fabdbb.slice/crio-89a567bbd82af87b4ae498f5512d129e72470c79cff81dcaa9cfc0aeb62d9db3 WatchSource:0}: Error finding container 89a567bbd82af87b4ae498f5512d129e72470c79cff81dcaa9cfc0aeb62d9db3: Status 404 returned error can't find the container with id 89a567bbd82af87b4ae498f5512d129e72470c79cff81dcaa9cfc0aeb62d9db3 Aug 03 16:05:17.369944 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:17.369903 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xwhgl" event={"ID":"bc59383f-9acf-44f5-85a0-fe7eb7189356","Type":"ContainerStarted","Data":"dbc6f4aecff7c7a25aba8b09fb28b072e1655d7720f4a8617a5ab2b045040ee9"} Aug 03 16:05:17.371396 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:17.371339 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" event={"ID":"d6f748fd-5da6-4f68-9786-0975f6fabdbb","Type":"ContainerStarted","Data":"89a567bbd82af87b4ae498f5512d129e72470c79cff81dcaa9cfc0aeb62d9db3"} Aug 03 16:05:17.374188 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:17.373412 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4xtzr" event={"ID":"b4167a63-fa6c-427a-8be0-51fb4be4540d","Type":"ContainerStarted","Data":"6851ecd3a19ea48f1e9e5c187c78bc5e20d1bd1b57cc6528622eaa0db23eece8"} Aug 03 16:05:17.376005 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:17.375980 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7hj45" event={"ID":"1024e841-024a-4af9-a182-c08ab00e2bf8","Type":"ContainerStarted","Data":"24102e805794db4e0fcf3b1e08cbf9c683442c9c36ee2547a8d5f2960740e7fa"} Aug 03 16:05:17.393691 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:17.393315 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4xtzr" podStartSLOduration=1.768132303 podStartE2EDuration="4.393288593s" totalImagesPullingTime="2.62515629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:05:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:17.39251659 +0000 UTC m=+55.950599175" watchObservedRunningTime="2026-08-03 16:05:17.393288593 +0000 UTC m=+55.951371176" Aug 03 16:05:17.410094 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:17.410044 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7hj45" podStartSLOduration=1.475130467 podStartE2EDuration="4.410028831s" totalImagesPullingTime="2.934898364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:17.409403115 +0000 UTC m=+55.967485697" watchObservedRunningTime="2026-08-03 16:05:17.410028831 +0000 UTC m=+55.968111415" Aug 03 16:05:18.384116 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:18.384081 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xwhgl" event={"ID":"bc59383f-9acf-44f5-85a0-fe7eb7189356","Type":"ContainerStarted","Data":"3e74f0f434936ab991d6803f9b35390b89698ea0bfe667ee2c2f7e4b744521d0"} Aug 03 16:05:18.384484 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:18.384347 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:18.402737 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:18.402697 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xwhgl" podStartSLOduration=3.048228649 podStartE2EDuration="5.402683998s" totalImagesPullingTime="2.354455349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:18.402554205 +0000 UTC m=+56.960636789" watchObservedRunningTime="2026-08-03 16:05:18.402683998 +0000 UTC m=+56.960766582" Aug 03 16:05:19.051960 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.051930 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-654c656ddf-fh54n"] Aug 03 16:05:19.073539 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.073514 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654c656ddf-fh54n"] Aug 03 16:05:19.073700 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.073641 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.076553 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.076502 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Aug 03 16:05:19.076692 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.076661 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5c8lp\"" Aug 03 16:05:19.077848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.077786 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Aug 03 16:05:19.077848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.077803 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Aug 03 16:05:19.077848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.077826 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Aug 03 16:05:19.077848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.077828 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 03 16:05:19.078235 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.078133 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 03 16:05:19.078467 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.078439 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Aug 03 16:05:19.180116 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.180079 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-service-ca\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.180116 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.180115 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-oauth-serving-cert\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.180313 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.180133 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-console-config\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.180313 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.180229 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-serving-cert\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.180313 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.180257 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-oauth-config\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.180313 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.180298 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4pmj\" (UniqueName: \"kubernetes.io/projected/a52b2559-1175-4905-b8d4-cb62d8709396-kube-api-access-p4pmj\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.281036 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.281005 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-service-ca\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.281036 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.281037 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-oauth-serving-cert\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.281233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.281056 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-console-config\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.281233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.281201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-serving-cert\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.281297 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.281240 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-oauth-config\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.281297 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.281286 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4pmj\" (UniqueName: \"kubernetes.io/projected/a52b2559-1175-4905-b8d4-cb62d8709396-kube-api-access-p4pmj\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.281825 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.281806 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-console-config\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.281921 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.281866 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-service-ca\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.281921 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.281866 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-oauth-serving-cert\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.283566 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.283545 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-oauth-config\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.283671 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.283644 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-serving-cert\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.290513 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.290487 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4pmj\" (UniqueName: \"kubernetes.io/projected/a52b2559-1175-4905-b8d4-cb62d8709396-kube-api-access-p4pmj\") pod \"console-654c656ddf-fh54n\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.382337 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.382267 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:19.388770 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.388742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" event={"ID":"d6f748fd-5da6-4f68-9786-0975f6fabdbb","Type":"ContainerStarted","Data":"4b41560a76f910f23cfc00d28cbd15f9d50be3287bfd2b19c159b7435d9fa46c"} Aug 03 16:05:19.389133 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.389101 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" Aug 03 16:05:19.393413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.393396 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" Aug 03 16:05:19.406603 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.406562 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-66685ff6d4-z4w99" podStartSLOduration=3.901242507 podStartE2EDuration="5.406550527s" totalImagesPullingTime="1.50530802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:19.405016374 +0000 UTC m=+57.963098958" watchObservedRunningTime="2026-08-03 16:05:19.406550527 +0000 UTC m=+57.964633110" Aug 03 16:05:19.527538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:19.527508 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654c656ddf-fh54n"] Aug 03 16:05:19.531275 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:19.531244 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52b2559_1175_4905_b8d4_cb62d8709396.slice/crio-37287ac905345339bc6666d64c3f136a23b35169d0956d4a9008c1ea1d9fc516 WatchSource:0}: Error finding container 37287ac905345339bc6666d64c3f136a23b35169d0956d4a9008c1ea1d9fc516: Status 404 returned error can't find the container with id 37287ac905345339bc6666d64c3f136a23b35169d0956d4a9008c1ea1d9fc516 Aug 03 16:05:20.393764 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:20.393731 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654c656ddf-fh54n" event={"ID":"a52b2559-1175-4905-b8d4-cb62d8709396","Type":"ContainerStarted","Data":"37287ac905345339bc6666d64c3f136a23b35169d0956d4a9008c1ea1d9fc516"} Aug 03 16:05:21.303413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:21.303383 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5x55b" Aug 03 16:05:23.406651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:23.406602 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654c656ddf-fh54n" event={"ID":"a52b2559-1175-4905-b8d4-cb62d8709396","Type":"ContainerStarted","Data":"784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d"} Aug 03 16:05:23.425760 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:23.425712 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-654c656ddf-fh54n" podStartSLOduration=1.3738810799999999 podStartE2EDuration="4.425696981s" totalImagesPullingTime="3.051815901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:23.425054008 +0000 UTC m=+61.983136592" watchObservedRunningTime="2026-08-03 16:05:23.425696981 +0000 UTC m=+61.983779565" Aug 03 16:05:25.208495 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.208462 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl"] Aug 03 16:05:25.211149 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.211134 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.214033 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.214015 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Aug 03 16:05:25.215141 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.215114 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Aug 03 16:05:25.215249 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.215148 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 03 16:05:25.215249 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.215153 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 03 16:05:25.215249 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.215225 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-sq5bj\"" Aug 03 16:05:25.215403 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.215394 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 03 16:05:25.217755 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.217704 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p5hcc"] Aug 03 16:05:25.219443 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.219426 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.222038 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.222017 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl"] Aug 03 16:05:25.222137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.222097 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 03 16:05:25.222197 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.222160 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 03 16:05:25.222197 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.222182 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 03 16:05:25.222290 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.222211 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8f8d9\"" Aug 03 16:05:25.242053 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.242028 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7"] Aug 03 16:05:25.244141 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.244122 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.246874 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.246855 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-krzm8\"" Aug 03 16:05:25.246979 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.246892 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Aug 03 16:05:25.246979 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.246901 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Aug 03 16:05:25.247668 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.247652 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Aug 03 16:05:25.255566 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.255548 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7"] Aug 03 16:05:25.329186 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329153 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g7lc\" (UniqueName: \"kubernetes.io/projected/a3b2df13-0345-49e4-a33b-2c367fcdce45-kube-api-access-2g7lc\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.329186 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329199 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.329410 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329229 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-tls\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.329410 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329274 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-wtmp\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.329410 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329335 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3b2df13-0345-49e4-a33b-2c367fcdce45-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.329410 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329371 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.329584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329418 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3b2df13-0345-49e4-a33b-2c367fcdce45-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.329584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329436 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-accelerators-collector-config\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.329584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329474 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65de89b0-fa4c-4b10-914e-a2a62557317e-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.329584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329519 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k7wc\" (UniqueName: \"kubernetes.io/projected/17618d89-6184-4883-a2a1-91c1b35c3625-kube-api-access-7k7wc\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.329735 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329603 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-textfile\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.329735 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329646 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.329735 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329683 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvvfd\" (UniqueName: \"kubernetes.io/projected/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-api-access-zvvfd\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.329735 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329702 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17618d89-6184-4883-a2a1-91c1b35c3625-sys\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.329881 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/65de89b0-fa4c-4b10-914e-a2a62557317e-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.329881 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329802 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.329881 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329821 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17618d89-6184-4883-a2a1-91c1b35c3625-metrics-client-ca\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.329881 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329842 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2df13-0345-49e4-a33b-2c367fcdce45-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.329881 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.329857 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17618d89-6184-4883-a2a1-91c1b35c3625-root\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.430842 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.430812 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/65de89b0-fa4c-4b10-914e-a2a62557317e-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.431031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.430864 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.431031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.430886 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17618d89-6184-4883-a2a1-91c1b35c3625-metrics-client-ca\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.431031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.430916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2df13-0345-49e4-a33b-2c367fcdce45-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.431031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.430939 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17618d89-6184-4883-a2a1-91c1b35c3625-root\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.431031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.430966 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g7lc\" (UniqueName: \"kubernetes.io/projected/a3b2df13-0345-49e4-a33b-2c367fcdce45-kube-api-access-2g7lc\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.431299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431055 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.431299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431132 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/17618d89-6184-4883-a2a1-91c1b35c3625-root\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.431299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431257 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-tls\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.431299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431271 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/65de89b0-fa4c-4b10-914e-a2a62557317e-volume-directive-shadow\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.431299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431291 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-wtmp\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.431547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431433 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-wtmp\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.431547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3b2df13-0345-49e4-a33b-2c367fcdce45-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.431547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.431547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431532 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3b2df13-0345-49e4-a33b-2c367fcdce45-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.431780 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431560 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-accelerators-collector-config\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.431780 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431603 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65de89b0-fa4c-4b10-914e-a2a62557317e-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.431780 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431660 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k7wc\" (UniqueName: \"kubernetes.io/projected/17618d89-6184-4883-a2a1-91c1b35c3625-kube-api-access-7k7wc\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.431780 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.431657 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17618d89-6184-4883-a2a1-91c1b35c3625-metrics-client-ca\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.432171 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432143 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-accelerators-collector-config\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.432265 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-textfile\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.432265 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432234 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.432373 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432270 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvvfd\" (UniqueName: \"kubernetes.io/projected/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-api-access-zvvfd\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.432373 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432323 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17618d89-6184-4883-a2a1-91c1b35c3625-sys\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.432373 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432338 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65de89b0-fa4c-4b10-914e-a2a62557317e-metrics-client-ca\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.432893 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432551 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.432893 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432561 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a3b2df13-0345-49e4-a33b-2c367fcdce45-metrics-client-ca\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.432893 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432678 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17618d89-6184-4883-a2a1-91c1b35c3625-sys\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.432893 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.432851 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-textfile\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.433881 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.433849 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-tls\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.434029 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.434011 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-state-metrics-tls\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.434096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.434024 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3b2df13-0345-49e4-a33b-2c367fcdce45-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.434378 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.434360 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17618d89-6184-4883-a2a1-91c1b35c3625-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.434529 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.434508 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a3b2df13-0345-49e4-a33b-2c367fcdce45-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.434750 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.434733 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.439809 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.439786 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g7lc\" (UniqueName: \"kubernetes.io/projected/a3b2df13-0345-49e4-a33b-2c367fcdce45-kube-api-access-2g7lc\") pod \"openshift-state-metrics-6bb4d7b4dd-8w9zl\" (UID: \"a3b2df13-0345-49e4-a33b-2c367fcdce45\") " pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.440215 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.440193 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvvfd\" (UniqueName: \"kubernetes.io/projected/65de89b0-fa4c-4b10-914e-a2a62557317e-kube-api-access-zvvfd\") pod \"kube-state-metrics-6bd9664c4-5j8r7\" (UID: \"65de89b0-fa4c-4b10-914e-a2a62557317e\") " pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.440567 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.440547 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k7wc\" (UniqueName: \"kubernetes.io/projected/17618d89-6184-4883-a2a1-91c1b35c3625-kube-api-access-7k7wc\") pod \"node-exporter-p5hcc\" (UID: \"17618d89-6184-4883-a2a1-91c1b35c3625\") " pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.520558 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.520530 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" Aug 03 16:05:25.528425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.528399 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p5hcc" Aug 03 16:05:25.538607 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:25.538578 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17618d89_6184_4883_a2a1_91c1b35c3625.slice/crio-496492109d18c4a414fdabcf81833f172dd5b6c0da08253452c24e81547c57e7 WatchSource:0}: Error finding container 496492109d18c4a414fdabcf81833f172dd5b6c0da08253452c24e81547c57e7: Status 404 returned error can't find the container with id 496492109d18c4a414fdabcf81833f172dd5b6c0da08253452c24e81547c57e7 Aug 03 16:05:25.556451 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.556424 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" Aug 03 16:05:25.657473 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.657440 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl"] Aug 03 16:05:25.663587 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:25.663564 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b2df13_0345_49e4_a33b_2c367fcdce45.slice/crio-6575ff8a3a810b62b5e4b47bf47182bf4d5c33427fe4939cae580873e60876f6 WatchSource:0}: Error finding container 6575ff8a3a810b62b5e4b47bf47182bf4d5c33427fe4939cae580873e60876f6: Status 404 returned error can't find the container with id 6575ff8a3a810b62b5e4b47bf47182bf4d5c33427fe4939cae580873e60876f6 Aug 03 16:05:25.698894 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:25.698868 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7"] Aug 03 16:05:25.707604 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:25.707579 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65de89b0_fa4c_4b10_914e_a2a62557317e.slice/crio-f5aae3e22bb7b9e40e3701cefdc7130becab352845e9b8c2f7699c6bf67291cf WatchSource:0}: Error finding container f5aae3e22bb7b9e40e3701cefdc7130becab352845e9b8c2f7699c6bf67291cf: Status 404 returned error can't find the container with id f5aae3e22bb7b9e40e3701cefdc7130becab352845e9b8c2f7699c6bf67291cf Aug 03 16:05:26.270249 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.270222 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 03 16:05:26.273638 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.273606 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.276665 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.276637 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Aug 03 16:05:26.276665 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.276660 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Aug 03 16:05:26.277082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.277063 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Aug 03 16:05:26.277190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.277065 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Aug 03 16:05:26.277410 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.277395 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Aug 03 16:05:26.277486 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.277447 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Aug 03 16:05:26.277669 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.277655 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Aug 03 16:05:26.277765 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.277664 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Aug 03 16:05:26.278095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.278075 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Aug 03 16:05:26.278226 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.278203 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j7xt7\"" Aug 03 16:05:26.290119 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.290092 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 03 16:05:26.340632 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340576 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.340867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340648 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.340867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-web-config\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.340867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340750 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnwz8\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-kube-api-access-cnwz8\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.340867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340807 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.340867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340840 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.340867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340861 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.341190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340881 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.341190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340929 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-out\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.341190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.340966 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.341190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.341002 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.341190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.341029 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.341190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.341052 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.418299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.418256 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" event={"ID":"a3b2df13-0345-49e4-a33b-2c367fcdce45","Type":"ContainerStarted","Data":"d7478edd4b1e31ad2f5bb8cb422849ebe2352d20bd2ed650ce51ba34a5c7a8d7"} Aug 03 16:05:26.418464 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.418329 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" event={"ID":"a3b2df13-0345-49e4-a33b-2c367fcdce45","Type":"ContainerStarted","Data":"0f09158c47358fbc895ba7be22566e2867b4d5742b6081fdb541cb0f86286aa7"} Aug 03 16:05:26.418464 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.418362 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" event={"ID":"a3b2df13-0345-49e4-a33b-2c367fcdce45","Type":"ContainerStarted","Data":"6575ff8a3a810b62b5e4b47bf47182bf4d5c33427fe4939cae580873e60876f6"} Aug 03 16:05:26.419577 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.419551 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" event={"ID":"65de89b0-fa4c-4b10-914e-a2a62557317e","Type":"ContainerStarted","Data":"f5aae3e22bb7b9e40e3701cefdc7130becab352845e9b8c2f7699c6bf67291cf"} Aug 03 16:05:26.420830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.420809 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p5hcc" event={"ID":"17618d89-6184-4883-a2a1-91c1b35c3625","Type":"ContainerStarted","Data":"496492109d18c4a414fdabcf81833f172dd5b6c0da08253452c24e81547c57e7"} Aug 03 16:05:26.442262 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442234 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-web-config\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442397 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnwz8\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-kube-api-access-cnwz8\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442466 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442466 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442452 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442570 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442471 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442570 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442497 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442570 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442524 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-out\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442570 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442550 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442822 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442591 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442822 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442634 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442822 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442664 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442822 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442714 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.442822 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.442753 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.443677 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:26.443401 2582 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Aug 03 16:05:26.443677 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:05:26.443482 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-main-tls podName:9f90bfdf-3d25-4440-9a03-517241fa8c9f nodeName:}" failed. No retries permitted until 2026-08-03 16:05:26.943455633 +0000 UTC m=+65.501538197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f") : secret "alertmanager-main-tls" not found Aug 03 16:05:26.443844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.443727 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.444173 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.443995 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.445424 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.445397 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-web-config\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.445991 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.445929 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.447030 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.446949 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.447167 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.447130 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-out\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.450003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.448694 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-volume\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.450003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.448860 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.450151 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.450104 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.450600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.450505 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.451767 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.451748 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.452511 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.452490 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnwz8\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-kube-api-access-cnwz8\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.644978 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.644943 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:26.647679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.647663 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 03 16:05:26.657913 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.657889 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4baf391b-a641-4c8e-a1f0-e70a089ef564-metrics-certs\") pod \"network-metrics-daemon-2dgwx\" (UID: \"4baf391b-a641-4c8e-a1f0-e70a089ef564\") " pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:26.745440 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.745400 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:26.748205 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.748186 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 03 16:05:26.758600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.758579 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16baf6d7-7c56-4d29-82bd-36c91c330974-original-pull-secret\") pod \"global-pull-secret-syncer-rq67m\" (UID: \"16baf6d7-7c56-4d29-82bd-36c91c330974\") " pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:26.846538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.846453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:26.849504 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.849481 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 03 16:05:26.860340 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.860317 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 03 16:05:26.871383 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.871364 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pkhn\" (UniqueName: \"kubernetes.io/projected/36267cd5-1552-4c0b-ba41-a025192a0ea8-kube-api-access-4pkhn\") pod \"network-check-target-p65bb\" (UID: \"36267cd5-1552-4c0b-ba41-a025192a0ea8\") " pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:26.898083 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.898057 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rq67m" Aug 03 16:05:26.906700 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.906679 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzpf\"" Aug 03 16:05:26.911132 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.911111 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xrdkc\"" Aug 03 16:05:26.914147 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.914128 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:26.919475 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.919420 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2dgwx" Aug 03 16:05:26.947705 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.947373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:26.951858 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:26.951823 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:27.050446 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.050418 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rq67m"] Aug 03 16:05:27.052229 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:27.052194 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16baf6d7_7c56_4d29_82bd_36c91c330974.slice/crio-008fcb4a172f76e2388a9006d9cba330f16ecd0ed555180a8129a10ad0a2ccc2 WatchSource:0}: Error finding container 008fcb4a172f76e2388a9006d9cba330f16ecd0ed555180a8129a10ad0a2ccc2: Status 404 returned error can't find the container with id 008fcb4a172f76e2388a9006d9cba330f16ecd0ed555180a8129a10ad0a2ccc2 Aug 03 16:05:27.080387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.080355 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p65bb"] Aug 03 16:05:27.096291 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.096270 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2dgwx"] Aug 03 16:05:27.185319 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.185283 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:05:27.261216 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:27.261184 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36267cd5_1552_4c0b_ba41_a025192a0ea8.slice/crio-00e3d09d2d01d37c5915683864f995187785cfb49f7d79693342535ba9cdad44 WatchSource:0}: Error finding container 00e3d09d2d01d37c5915683864f995187785cfb49f7d79693342535ba9cdad44: Status 404 returned error can't find the container with id 00e3d09d2d01d37c5915683864f995187785cfb49f7d79693342535ba9cdad44 Aug 03 16:05:27.261908 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:27.261885 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4baf391b_a641_4c8e_a1f0_e70a089ef564.slice/crio-269d2c16393840a6ca0a387b21f1efff0cc4fa4d9edc01017ae79b2fbd320647 WatchSource:0}: Error finding container 269d2c16393840a6ca0a387b21f1efff0cc4fa4d9edc01017ae79b2fbd320647: Status 404 returned error can't find the container with id 269d2c16393840a6ca0a387b21f1efff0cc4fa4d9edc01017ae79b2fbd320647 Aug 03 16:05:27.391428 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.391400 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 03 16:05:27.394253 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:27.394224 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f90bfdf_3d25_4440_9a03_517241fa8c9f.slice/crio-ce46de9a3769d0b971f68278bbefd3d8a843022fd228471ba61778bf425d0bf8 WatchSource:0}: Error finding container ce46de9a3769d0b971f68278bbefd3d8a843022fd228471ba61778bf425d0bf8: Status 404 returned error can't find the container with id ce46de9a3769d0b971f68278bbefd3d8a843022fd228471ba61778bf425d0bf8 Aug 03 16:05:27.425639 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.425587 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" event={"ID":"a3b2df13-0345-49e4-a33b-2c367fcdce45","Type":"ContainerStarted","Data":"309c280850e9025bb0fd93d6e84d97de30ba572771caa7a9d2f4d1b12d83a7b0"} Aug 03 16:05:27.426789 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.426765 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerStarted","Data":"ce46de9a3769d0b971f68278bbefd3d8a843022fd228471ba61778bf425d0bf8"} Aug 03 16:05:27.427852 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.427819 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p65bb" event={"ID":"36267cd5-1552-4c0b-ba41-a025192a0ea8","Type":"ContainerStarted","Data":"00e3d09d2d01d37c5915683864f995187785cfb49f7d79693342535ba9cdad44"} Aug 03 16:05:27.428864 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.428844 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rq67m" event={"ID":"16baf6d7-7c56-4d29-82bd-36c91c330974","Type":"ContainerStarted","Data":"008fcb4a172f76e2388a9006d9cba330f16ecd0ed555180a8129a10ad0a2ccc2"} Aug 03 16:05:27.430256 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.430235 2582 generic.go:356] "Generic (PLEG): container finished" podID="17618d89-6184-4883-a2a1-91c1b35c3625" containerID="e315db8593bd2ec00b9c94a347feca4b2031df36da1ee1d79e9a828db35c73ff" exitCode=0 Aug 03 16:05:27.430356 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.430305 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p5hcc" event={"ID":"17618d89-6184-4883-a2a1-91c1b35c3625","Type":"ContainerDied","Data":"e315db8593bd2ec00b9c94a347feca4b2031df36da1ee1d79e9a828db35c73ff"} Aug 03 16:05:27.431295 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.431277 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2dgwx" event={"ID":"4baf391b-a641-4c8e-a1f0-e70a089ef564","Type":"ContainerStarted","Data":"269d2c16393840a6ca0a387b21f1efff0cc4fa4d9edc01017ae79b2fbd320647"} Aug 03 16:05:27.443923 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:27.443884 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6bb4d7b4dd-8w9zl" podStartSLOduration=0.908285536 podStartE2EDuration="2.443870926s" totalImagesPullingTime="1.53558539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:27.44334905 +0000 UTC m=+66.001431637" watchObservedRunningTime="2026-08-03 16:05:27.443870926 +0000 UTC m=+66.001953512" Aug 03 16:05:28.197967 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.197933 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-668d95d56c-862fz"] Aug 03 16:05:28.201741 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.201712 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.205363 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.205339 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rgmft\"" Aug 03 16:05:28.205911 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.205887 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Aug 03 16:05:28.206018 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.205979 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Aug 03 16:05:28.206087 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.206014 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Aug 03 16:05:28.206266 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.206246 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Aug 03 16:05:28.206636 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.206605 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-d3rbpnl8ef3i9\"" Aug 03 16:05:28.211272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.211250 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Aug 03 16:05:28.218785 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.218760 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-668d95d56c-862fz"] Aug 03 16:05:28.359253 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.359127 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-grpc-tls\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.359253 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.359193 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.359496 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.359289 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.359496 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.359327 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.359496 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.359402 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8xzv\" (UniqueName: \"kubernetes.io/projected/bfa9e619-6beb-4f50-94ce-bb403aa164b6-kube-api-access-f8xzv\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.359496 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.359430 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-tls\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.359496 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.359463 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfa9e619-6beb-4f50-94ce-bb403aa164b6-metrics-client-ca\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.359777 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.359507 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.391172 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.391142 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xwhgl" Aug 03 16:05:28.438728 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.438678 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p5hcc" event={"ID":"17618d89-6184-4883-a2a1-91c1b35c3625","Type":"ContainerStarted","Data":"772ec0c4949a112dc3c62745f879b2129375a06d1b9bf33eef9f7f372ec6b7e9"} Aug 03 16:05:28.439167 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.438736 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p5hcc" event={"ID":"17618d89-6184-4883-a2a1-91c1b35c3625","Type":"ContainerStarted","Data":"cffbc9008772db0fdc0879797ecfb0a82c9d44c54f9d8c4a525d59bb8c2ca008"} Aug 03 16:05:28.461890 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.461362 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.461890 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.461417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.461890 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.461460 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.461890 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.461536 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8xzv\" (UniqueName: \"kubernetes.io/projected/bfa9e619-6beb-4f50-94ce-bb403aa164b6-kube-api-access-f8xzv\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.461890 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.461564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-tls\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.461890 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.461607 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfa9e619-6beb-4f50-94ce-bb403aa164b6-metrics-client-ca\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.461890 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.461666 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.461890 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.461718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-grpc-tls\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.465405 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.465350 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bfa9e619-6beb-4f50-94ce-bb403aa164b6-metrics-client-ca\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.473721 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.468941 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p5hcc" podStartSLOduration=2.466796852 podStartE2EDuration="3.468908128s" totalImagesPullingTime="938.350903ms" totalInitContainerRuntime="63.760373ms" isStatefulPod=true podCreationTimestamp="2026-08-03 16:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:28.460299007 +0000 UTC m=+67.018381612" watchObservedRunningTime="2026-08-03 16:05:28.468908128 +0000 UTC m=+67.026990694" Aug 03 16:05:28.473721 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.475386 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-tls\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.476527 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.476486 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.481864 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.481817 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.482930 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.482909 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8xzv\" (UniqueName: \"kubernetes.io/projected/bfa9e619-6beb-4f50-94ce-bb403aa164b6-kube-api-access-f8xzv\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.483100 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.483040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-grpc-tls\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.483100 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.482948 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.484081 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.484059 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bfa9e619-6beb-4f50-94ce-bb403aa164b6-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668d95d56c-862fz\" (UID: \"bfa9e619-6beb-4f50-94ce-bb403aa164b6\") " pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.513003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.512919 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:28.782833 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:28.782009 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-668d95d56c-862fz"] Aug 03 16:05:28.795330 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:28.795292 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa9e619_6beb_4f50_94ce_bb403aa164b6.slice/crio-7e8de0099ed8acc609d0e0e97852a0df88f8bee15c4a7aa34ba1b0529b0c0aaa WatchSource:0}: Error finding container 7e8de0099ed8acc609d0e0e97852a0df88f8bee15c4a7aa34ba1b0529b0c0aaa: Status 404 returned error can't find the container with id 7e8de0099ed8acc609d0e0e97852a0df88f8bee15c4a7aa34ba1b0529b0c0aaa Aug 03 16:05:29.383021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.382986 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:29.383182 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.383054 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:29.389744 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.389468 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:29.446408 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.445813 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" event={"ID":"65de89b0-fa4c-4b10-914e-a2a62557317e","Type":"ContainerStarted","Data":"d553d213780b711f97d87947c55602e3f8d5ccb1b0b658c4f5e6ca6ea2e940be"} Aug 03 16:05:29.446408 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.445878 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" event={"ID":"65de89b0-fa4c-4b10-914e-a2a62557317e","Type":"ContainerStarted","Data":"25616dbcf39e633761e2c9b910f1077b94c854c37a663d1605fe0b8e5eeb8661"} Aug 03 16:05:29.446408 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.445910 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" event={"ID":"65de89b0-fa4c-4b10-914e-a2a62557317e","Type":"ContainerStarted","Data":"7fdf26d8479dae7f98754125ee4e02ff6e2bfa7f6ee5891c00b0e41e8eda44ce"} Aug 03 16:05:29.448150 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.448117 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2dgwx" event={"ID":"4baf391b-a641-4c8e-a1f0-e70a089ef564","Type":"ContainerStarted","Data":"a24966c93389e45a285bfa337ec1bff8b23be27eac9764eda53993eeefb0a092"} Aug 03 16:05:29.448270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.448170 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2dgwx" event={"ID":"4baf391b-a641-4c8e-a1f0-e70a089ef564","Type":"ContainerStarted","Data":"bce4b306d24ec36a90a85a40c93b6ff3d1ed41c37c36482ae9ef408e05155482"} Aug 03 16:05:29.449953 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.449929 2582 generic.go:356] "Generic (PLEG): container finished" podID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerID="e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188" exitCode=0 Aug 03 16:05:29.450052 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.449967 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerDied","Data":"e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188"} Aug 03 16:05:29.452409 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.452389 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" event={"ID":"bfa9e619-6beb-4f50-94ce-bb403aa164b6","Type":"ContainerStarted","Data":"7e8de0099ed8acc609d0e0e97852a0df88f8bee15c4a7aa34ba1b0529b0c0aaa"} Aug 03 16:05:29.458140 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.458065 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:05:29.466145 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.466098 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-6bd9664c4-5j8r7" podStartSLOduration=1.568706776 podStartE2EDuration="4.466080728s" totalImagesPullingTime="2.897373952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:29.464842448 +0000 UTC m=+68.022925035" watchObservedRunningTime="2026-08-03 16:05:29.466080728 +0000 UTC m=+68.024163316" Aug 03 16:05:29.529516 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:29.529452 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2dgwx" podStartSLOduration=66.184013236 podStartE2EDuration="1m7.529429153s" totalImagesPullingTime="1.345415917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:29.528600889 +0000 UTC m=+68.086683476" watchObservedRunningTime="2026-08-03 16:05:29.529429153 +0000 UTC m=+68.087511740" Aug 03 16:05:32.277554 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.277516 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74d6b7b77c-5mw9s"] Aug 03 16:05:32.281466 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.281447 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.292891 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.292832 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Aug 03 16:05:32.293321 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.293263 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74d6b7b77c-5mw9s"] Aug 03 16:05:32.402168 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.402139 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f92h9\" (UniqueName: \"kubernetes.io/projected/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-kube-api-access-f92h9\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.402334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.402178 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-trusted-ca-bundle\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.402334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.402213 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-oauth-serving-cert\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.402334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.402237 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-service-ca\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.402334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.402285 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-serving-cert\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.402334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.402307 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-config\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.402334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.402337 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-oauth-config\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.503497 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.503460 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f92h9\" (UniqueName: \"kubernetes.io/projected/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-kube-api-access-f92h9\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.503720 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.503508 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-trusted-ca-bundle\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.503720 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.503535 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-oauth-serving-cert\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.503720 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.503592 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-service-ca\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.503720 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.503682 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-serving-cert\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.503720 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.503707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-config\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.503960 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.503735 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-oauth-config\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.504910 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.504881 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-oauth-serving-cert\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.505017 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.504961 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-service-ca\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.505017 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.504971 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-config\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.505134 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.505115 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-trusted-ca-bundle\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.506629 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.506595 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-oauth-config\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.506793 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.506775 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-serving-cert\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.513096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.513059 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f92h9\" (UniqueName: \"kubernetes.io/projected/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-kube-api-access-f92h9\") pod \"console-74d6b7b77c-5mw9s\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:32.594699 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:32.594598 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:33.630852 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:33.630725 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74d6b7b77c-5mw9s"] Aug 03 16:05:33.637243 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:05:33.637201 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e7da64_796c_4475_aa2a_f54d3a7a9ef8.slice/crio-2224638636f0d9693c5abf2adba67590f3cbbbeb3f69c3a61bcb6ae5d1f24371 WatchSource:0}: Error finding container 2224638636f0d9693c5abf2adba67590f3cbbbeb3f69c3a61bcb6ae5d1f24371: Status 404 returned error can't find the container with id 2224638636f0d9693c5abf2adba67590f3cbbbeb3f69c3a61bcb6ae5d1f24371 Aug 03 16:05:34.475532 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.475501 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" event={"ID":"bfa9e619-6beb-4f50-94ce-bb403aa164b6","Type":"ContainerStarted","Data":"d8b4c903bcb62d9cf8c8609f87d6cf8d9c30a6f1948a4f10edbc0df2d55eb3b1"} Aug 03 16:05:34.475707 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.475583 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" event={"ID":"bfa9e619-6beb-4f50-94ce-bb403aa164b6","Type":"ContainerStarted","Data":"fb9a816c24a44d8b0faeb9f66caaca4870947f9ec1690683b1d1a986338a88e9"} Aug 03 16:05:34.475707 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.475645 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" event={"ID":"bfa9e619-6beb-4f50-94ce-bb403aa164b6","Type":"ContainerStarted","Data":"0c80878bf109d21d7a90c1021a3cf663635375e38deeef219bae2982da8e7226"} Aug 03 16:05:34.477004 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.476982 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d6b7b77c-5mw9s" event={"ID":"26e7da64-796c-4475-aa2a-f54d3a7a9ef8","Type":"ContainerStarted","Data":"feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7"} Aug 03 16:05:34.477075 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.477031 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d6b7b77c-5mw9s" event={"ID":"26e7da64-796c-4475-aa2a-f54d3a7a9ef8","Type":"ContainerStarted","Data":"2224638636f0d9693c5abf2adba67590f3cbbbeb3f69c3a61bcb6ae5d1f24371"} Aug 03 16:05:34.479940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.479917 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerStarted","Data":"f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4"} Aug 03 16:05:34.480050 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.479974 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerStarted","Data":"2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4"} Aug 03 16:05:34.480050 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.480007 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerStarted","Data":"9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b"} Aug 03 16:05:34.480050 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.480039 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerStarted","Data":"74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416"} Aug 03 16:05:34.480192 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.480077 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerStarted","Data":"8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db"} Aug 03 16:05:34.481391 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.481367 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p65bb" event={"ID":"36267cd5-1552-4c0b-ba41-a025192a0ea8","Type":"ContainerStarted","Data":"d5a9831754cf448c478c2599b094c6bd0aec6cbf715e2f0780e6f39df4cc82f3"} Aug 03 16:05:34.481502 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.481430 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:05:34.482843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.482820 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rq67m" event={"ID":"16baf6d7-7c56-4d29-82bd-36c91c330974","Type":"ContainerStarted","Data":"d5079ba2282a0cbe5a19472095ab67cfaf9be5e9c1be67bb32128a40950c6162"} Aug 03 16:05:34.498885 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.498835 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74d6b7b77c-5mw9s" podStartSLOduration=2.49882038 podStartE2EDuration="2.49882038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:34.498080677 +0000 UTC m=+73.056163264" watchObservedRunningTime="2026-08-03 16:05:34.49882038 +0000 UTC m=+73.056902944" Aug 03 16:05:34.514491 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.514443 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rq67m" podStartSLOduration=66.109927183 podStartE2EDuration="1m12.514425005s" totalImagesPullingTime="6.404497822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:34.513532608 +0000 UTC m=+73.071615195" watchObservedRunningTime="2026-08-03 16:05:34.514425005 +0000 UTC m=+73.072507594" Aug 03 16:05:34.529563 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:34.529522 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p65bb" podStartSLOduration=66.334907755 podStartE2EDuration="1m12.52950876s" totalImagesPullingTime="6.194601005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:04:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:34.528870519 +0000 UTC m=+73.086953126" watchObservedRunningTime="2026-08-03 16:05:34.52950876 +0000 UTC m=+73.087591345" Aug 03 16:05:35.490265 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:35.490161 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" event={"ID":"bfa9e619-6beb-4f50-94ce-bb403aa164b6","Type":"ContainerStarted","Data":"faca06443a70cb6ea577b62187e60a9e314b8f8c46f416138520d7d256732efa"} Aug 03 16:05:35.490747 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:35.490273 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" event={"ID":"bfa9e619-6beb-4f50-94ce-bb403aa164b6","Type":"ContainerStarted","Data":"1a3f9dd0719940fbb61b28035938e4dc47c500922a71f0183032bbe44fc988aa"} Aug 03 16:05:35.490747 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:35.490313 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" event={"ID":"bfa9e619-6beb-4f50-94ce-bb403aa164b6","Type":"ContainerStarted","Data":"8360d90ab04ad342d8bb22e42d17dc2ecd282f91467324dd95f602bcc0f8d431"} Aug 03 16:05:35.490747 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:35.490354 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:35.493227 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:35.493197 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerStarted","Data":"990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603"} Aug 03 16:05:35.515546 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:35.515498 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" podStartSLOduration=2.091338908 podStartE2EDuration="7.515479881s" totalImagesPullingTime="5.424140973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:35.513806171 +0000 UTC m=+74.071888767" watchObservedRunningTime="2026-08-03 16:05:35.515479881 +0000 UTC m=+74.073562467" Aug 03 16:05:35.541925 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:35.541875 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.268692386 podStartE2EDuration="9.541861013s" totalImagesPullingTime="7.254467659s" totalInitContainerRuntime="18.700968ms" isStatefulPod=false podCreationTimestamp="2026-08-03 16:05:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:05:35.541082053 +0000 UTC m=+74.099164640" watchObservedRunningTime="2026-08-03 16:05:35.541861013 +0000 UTC m=+74.099943598" Aug 03 16:05:41.505338 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:41.505309 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-668d95d56c-862fz" Aug 03 16:05:42.594966 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:42.594918 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:42.595321 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:42.595025 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:42.599768 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:42.599746 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:43.536013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:43.535906 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:05:43.587428 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:05:43.587395 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654c656ddf-fh54n"] Aug 03 16:06:05.496397 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:05.496366 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p65bb" Aug 03 16:06:08.606783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.606740 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-654c656ddf-fh54n" podUID="a52b2559-1175-4905-b8d4-cb62d8709396" containerName="console" containerID="cri-o://784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d" gracePeriod=15 Aug 03 16:06:08.845339 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.845317 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:06:08.888360 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.888269 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-oauth-config\") pod \"a52b2559-1175-4905-b8d4-cb62d8709396\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " Aug 03 16:06:08.888360 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.888311 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-serving-cert\") pod \"a52b2559-1175-4905-b8d4-cb62d8709396\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " Aug 03 16:06:08.888360 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.888346 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-oauth-serving-cert\") pod \"a52b2559-1175-4905-b8d4-cb62d8709396\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " Aug 03 16:06:08.888656 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.888397 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-console-config\") pod \"a52b2559-1175-4905-b8d4-cb62d8709396\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " Aug 03 16:06:08.888656 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.888419 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-service-ca\") pod \"a52b2559-1175-4905-b8d4-cb62d8709396\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " Aug 03 16:06:08.888656 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.888489 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4pmj\" (UniqueName: \"kubernetes.io/projected/a52b2559-1175-4905-b8d4-cb62d8709396-kube-api-access-p4pmj\") pod \"a52b2559-1175-4905-b8d4-cb62d8709396\" (UID: \"a52b2559-1175-4905-b8d4-cb62d8709396\") " Aug 03 16:06:08.888863 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.888834 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a52b2559-1175-4905-b8d4-cb62d8709396" (UID: "a52b2559-1175-4905-b8d4-cb62d8709396"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:06:08.888863 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.888842 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-service-ca" (OuterVolumeSpecName: "service-ca") pod "a52b2559-1175-4905-b8d4-cb62d8709396" (UID: "a52b2559-1175-4905-b8d4-cb62d8709396"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:06:08.888970 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.888830 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-console-config" (OuterVolumeSpecName: "console-config") pod "a52b2559-1175-4905-b8d4-cb62d8709396" (UID: "a52b2559-1175-4905-b8d4-cb62d8709396"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:06:08.890819 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.890778 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a52b2559-1175-4905-b8d4-cb62d8709396" (UID: "a52b2559-1175-4905-b8d4-cb62d8709396"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:06:08.890938 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.890884 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a52b2559-1175-4905-b8d4-cb62d8709396" (UID: "a52b2559-1175-4905-b8d4-cb62d8709396"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:06:08.890938 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.890897 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52b2559-1175-4905-b8d4-cb62d8709396-kube-api-access-p4pmj" (OuterVolumeSpecName: "kube-api-access-p4pmj") pod "a52b2559-1175-4905-b8d4-cb62d8709396" (UID: "a52b2559-1175-4905-b8d4-cb62d8709396"). InnerVolumeSpecName "kube-api-access-p4pmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:06:08.989783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.989750 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-console-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:08.989783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.989776 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-service-ca\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:08.989783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.989785 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4pmj\" (UniqueName: \"kubernetes.io/projected/a52b2559-1175-4905-b8d4-cb62d8709396-kube-api-access-p4pmj\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:08.989783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.989795 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-oauth-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:08.990040 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.989804 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52b2559-1175-4905-b8d4-cb62d8709396-console-serving-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:08.990040 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:08.989840 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52b2559-1175-4905-b8d4-cb62d8709396-oauth-serving-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:09.627183 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.627143 2582 generic.go:356] "Generic (PLEG): container finished" podID="a52b2559-1175-4905-b8d4-cb62d8709396" containerID="784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d" exitCode=0 Aug 03 16:06:09.627183 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.627188 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654c656ddf-fh54n" event={"ID":"a52b2559-1175-4905-b8d4-cb62d8709396","Type":"ContainerDied","Data":"784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d"} Aug 03 16:06:09.627748 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.627191 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654c656ddf-fh54n" Aug 03 16:06:09.627748 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.627229 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654c656ddf-fh54n" event={"ID":"a52b2559-1175-4905-b8d4-cb62d8709396","Type":"ContainerDied","Data":"37287ac905345339bc6666d64c3f136a23b35169d0956d4a9008c1ea1d9fc516"} Aug 03 16:06:09.627748 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.627254 2582 scope.go:117] "RemoveContainer" containerID="784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d" Aug 03 16:06:09.635762 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.635739 2582 scope.go:117] "RemoveContainer" containerID="784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d" Aug 03 16:06:09.636008 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:06:09.635988 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d\": container with ID starting with 784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d not found: ID does not exist" containerID="784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d" Aug 03 16:06:09.636075 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.636021 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d"} err="failed to get container status \"784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d\": rpc error: code = NotFound desc = could not find container \"784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d\": container with ID starting with 784e44b4437c130670caca7f900f3b85df0c8fe570c98d5382b4c9257c1c116d not found: ID does not exist" Aug 03 16:06:09.648939 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.648909 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654c656ddf-fh54n"] Aug 03 16:06:09.651141 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.651117 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-654c656ddf-fh54n"] Aug 03 16:06:09.994453 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:09.994355 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a52b2559-1175-4905-b8d4-cb62d8709396" path="/var/lib/kubelet/pods/a52b2559-1175-4905-b8d4-cb62d8709396/volumes" Aug 03 16:06:45.493091 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.493061 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 03 16:06:45.493581 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.493365 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="alertmanager" containerID="cri-o://8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db" gracePeriod=120 Aug 03 16:06:45.493581 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.493392 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy" containerID="cri-o://2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4" gracePeriod=120 Aug 03 16:06:45.493581 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.493445 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="config-reloader" containerID="cri-o://74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416" gracePeriod=120 Aug 03 16:06:45.493581 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.493412 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="prom-label-proxy" containerID="cri-o://990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603" gracePeriod=120 Aug 03 16:06:45.493581 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.493443 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy-web" containerID="cri-o://9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b" gracePeriod=120 Aug 03 16:06:45.493581 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.493451 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy-metric" containerID="cri-o://f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4" gracePeriod=120 Aug 03 16:06:45.762548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.762459 2582 generic.go:356] "Generic (PLEG): container finished" podID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerID="990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603" exitCode=0 Aug 03 16:06:45.762548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.762485 2582 generic.go:356] "Generic (PLEG): container finished" podID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerID="2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4" exitCode=0 Aug 03 16:06:45.762548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.762491 2582 generic.go:356] "Generic (PLEG): container finished" podID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerID="74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416" exitCode=0 Aug 03 16:06:45.762548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.762497 2582 generic.go:356] "Generic (PLEG): container finished" podID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerID="8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db" exitCode=0 Aug 03 16:06:45.762548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.762534 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerDied","Data":"990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603"} Aug 03 16:06:45.762830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.762632 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerDied","Data":"2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4"} Aug 03 16:06:45.762830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.762676 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerDied","Data":"74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416"} Aug 03 16:06:45.762830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:45.762710 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerDied","Data":"8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db"} Aug 03 16:06:46.734839 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.734817 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:46.776246 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.776214 2582 generic.go:356] "Generic (PLEG): container finished" podID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerID="f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4" exitCode=0 Aug 03 16:06:46.776246 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.776239 2582 generic.go:356] "Generic (PLEG): container finished" podID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerID="9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b" exitCode=0 Aug 03 16:06:46.776544 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.776295 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerDied","Data":"f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4"} Aug 03 16:06:46.776544 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.776324 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:46.776544 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.776391 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerDied","Data":"9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b"} Aug 03 16:06:46.776544 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.776422 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9f90bfdf-3d25-4440-9a03-517241fa8c9f","Type":"ContainerDied","Data":"ce46de9a3769d0b971f68278bbefd3d8a843022fd228471ba61778bf425d0bf8"} Aug 03 16:06:46.776544 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.776460 2582 scope.go:117] "RemoveContainer" containerID="990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603" Aug 03 16:06:46.785314 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.785278 2582 scope.go:117] "RemoveContainer" containerID="f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4" Aug 03 16:06:46.792053 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.792028 2582 scope.go:117] "RemoveContainer" containerID="2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4" Aug 03 16:06:46.799094 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.799073 2582 scope.go:117] "RemoveContainer" containerID="9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b" Aug 03 16:06:46.805746 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.805726 2582 scope.go:117] "RemoveContainer" containerID="74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416" Aug 03 16:06:46.812672 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.812651 2582 scope.go:117] "RemoveContainer" containerID="8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db" Aug 03 16:06:46.819711 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.819690 2582 scope.go:117] "RemoveContainer" containerID="e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188" Aug 03 16:06:46.826156 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.826132 2582 scope.go:117] "RemoveContainer" containerID="990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603" Aug 03 16:06:46.826399 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:06:46.826381 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603\": container with ID starting with 990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603 not found: ID does not exist" containerID="990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603" Aug 03 16:06:46.826465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.826409 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603"} err="failed to get container status \"990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603\": rpc error: code = NotFound desc = could not find container \"990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603\": container with ID starting with 990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603 not found: ID does not exist" Aug 03 16:06:46.826465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.826426 2582 scope.go:117] "RemoveContainer" containerID="f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4" Aug 03 16:06:46.826748 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:06:46.826726 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4\": container with ID starting with f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4 not found: ID does not exist" containerID="f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4" Aug 03 16:06:46.826827 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.826757 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4"} err="failed to get container status \"f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4\": rpc error: code = NotFound desc = could not find container \"f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4\": container with ID starting with f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4 not found: ID does not exist" Aug 03 16:06:46.826827 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.826781 2582 scope.go:117] "RemoveContainer" containerID="2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4" Aug 03 16:06:46.827017 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:06:46.826997 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4\": container with ID starting with 2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4 not found: ID does not exist" containerID="2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4" Aug 03 16:06:46.827057 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827024 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4"} err="failed to get container status \"2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4\": rpc error: code = NotFound desc = could not find container \"2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4\": container with ID starting with 2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4 not found: ID does not exist" Aug 03 16:06:46.827057 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827044 2582 scope.go:117] "RemoveContainer" containerID="9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b" Aug 03 16:06:46.827283 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:06:46.827266 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b\": container with ID starting with 9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b not found: ID does not exist" containerID="9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b" Aug 03 16:06:46.827346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827290 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b"} err="failed to get container status \"9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b\": rpc error: code = NotFound desc = could not find container \"9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b\": container with ID starting with 9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b not found: ID does not exist" Aug 03 16:06:46.827346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827310 2582 scope.go:117] "RemoveContainer" containerID="74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416" Aug 03 16:06:46.827521 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:06:46.827506 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416\": container with ID starting with 74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416 not found: ID does not exist" containerID="74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416" Aug 03 16:06:46.827561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827524 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416"} err="failed to get container status \"74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416\": rpc error: code = NotFound desc = could not find container \"74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416\": container with ID starting with 74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416 not found: ID does not exist" Aug 03 16:06:46.827561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827540 2582 scope.go:117] "RemoveContainer" containerID="8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db" Aug 03 16:06:46.827750 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:06:46.827734 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db\": container with ID starting with 8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db not found: ID does not exist" containerID="8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db" Aug 03 16:06:46.827798 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827754 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db"} err="failed to get container status \"8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db\": rpc error: code = NotFound desc = could not find container \"8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db\": container with ID starting with 8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db not found: ID does not exist" Aug 03 16:06:46.827798 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827767 2582 scope.go:117] "RemoveContainer" containerID="e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188" Aug 03 16:06:46.827947 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:06:46.827930 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188\": container with ID starting with e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188 not found: ID does not exist" containerID="e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188" Aug 03 16:06:46.828007 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827955 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188"} err="failed to get container status \"e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188\": rpc error: code = NotFound desc = could not find container \"e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188\": container with ID starting with e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188 not found: ID does not exist" Aug 03 16:06:46.828007 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.827973 2582 scope.go:117] "RemoveContainer" containerID="990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603" Aug 03 16:06:46.828207 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.828190 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603"} err="failed to get container status \"990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603\": rpc error: code = NotFound desc = could not find container \"990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603\": container with ID starting with 990f998f4b76b1091f04dca23263cd7f9c0f3fa17b9622e37dfa4c024c064603 not found: ID does not exist" Aug 03 16:06:46.828255 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.828207 2582 scope.go:117] "RemoveContainer" containerID="f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4" Aug 03 16:06:46.828407 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.828386 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4"} err="failed to get container status \"f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4\": rpc error: code = NotFound desc = could not find container \"f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4\": container with ID starting with f298d7636cc8bc2b0790cb36692f41355879f903c4d8565702cf0d662e46b4c4 not found: ID does not exist" Aug 03 16:06:46.828407 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.828407 2582 scope.go:117] "RemoveContainer" containerID="2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4" Aug 03 16:06:46.828636 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.828604 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4"} err="failed to get container status \"2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4\": rpc error: code = NotFound desc = could not find container \"2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4\": container with ID starting with 2973521a5197e402e33eb75f0a96b718ece79afaaf904bc5484d8ffdd81ab7f4 not found: ID does not exist" Aug 03 16:06:46.828636 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.828633 2582 scope.go:117] "RemoveContainer" containerID="9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b" Aug 03 16:06:46.828839 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.828823 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b"} err="failed to get container status \"9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b\": rpc error: code = NotFound desc = could not find container \"9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b\": container with ID starting with 9e6bd6a480883a61a69e5513e9227b533ec29a3dc825209c26240112ba2f6f5b not found: ID does not exist" Aug 03 16:06:46.828839 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.828838 2582 scope.go:117] "RemoveContainer" containerID="74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416" Aug 03 16:06:46.829013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.828996 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416"} err="failed to get container status \"74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416\": rpc error: code = NotFound desc = could not find container \"74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416\": container with ID starting with 74d2be7c639947e2aaad57655174835dc670f99b0bc360d98e84c60900d04416 not found: ID does not exist" Aug 03 16:06:46.829062 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.829013 2582 scope.go:117] "RemoveContainer" containerID="8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db" Aug 03 16:06:46.829193 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.829177 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db"} err="failed to get container status \"8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db\": rpc error: code = NotFound desc = could not find container \"8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db\": container with ID starting with 8d46e915e318c8e86447a8ea4a6fefaa57a097a9ccdf1cba46bb1a101cef58db not found: ID does not exist" Aug 03 16:06:46.829238 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.829192 2582 scope.go:117] "RemoveContainer" containerID="e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188" Aug 03 16:06:46.829413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.829398 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188"} err="failed to get container status \"e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188\": rpc error: code = NotFound desc = could not find container \"e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188\": container with ID starting with e35c707b49f281194c8c86ef95569d7d7ecbeb499c3188644b51156716062188 not found: ID does not exist" Aug 03 16:06:46.902375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902347 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-main-db\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902535 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902381 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-cluster-tls-config\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902535 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902411 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902535 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902468 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-tls-assets\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902535 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902493 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-out\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902535 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902528 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-main-tls\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902829 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902686 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-volume\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902829 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902750 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-trusted-ca-bundle\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902829 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902755 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:06:46.902983 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902834 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902983 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902890 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnwz8\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-kube-api-access-cnwz8\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.902983 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902925 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-metrics-client-ca\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.903135 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.902950 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-web\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.903135 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.903052 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-web-config\") pod \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\" (UID: \"9f90bfdf-3d25-4440-9a03-517241fa8c9f\") " Aug 03 16:06:46.903228 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.903149 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:06:46.903600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.903314 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-main-db\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:46.903600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.903341 2582 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:46.904037 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.904011 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:06:46.905876 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.905845 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:06:46.905990 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.905921 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:06:46.906722 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.906697 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-out" (OuterVolumeSpecName: "config-out") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:06:46.906940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.906917 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:06:46.907020 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.906992 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-kube-api-access-cnwz8" (OuterVolumeSpecName: "kube-api-access-cnwz8") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "kube-api-access-cnwz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:06:46.907070 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.907032 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:06:46.907151 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.907129 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-volume" (OuterVolumeSpecName: "config-volume") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:06:46.907267 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.907250 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:06:46.910062 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.910039 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:06:46.916351 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:46.916328 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-web-config" (OuterVolumeSpecName: "web-config") pod "9f90bfdf-3d25-4440-9a03-517241fa8c9f" (UID: "9f90bfdf-3d25-4440-9a03-517241fa8c9f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:06:47.004193 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004144 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-main-tls\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004193 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004186 2582 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-volume\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004193 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004198 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004193 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004207 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnwz8\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-kube-api-access-cnwz8\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004193 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004215 2582 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f90bfdf-3d25-4440-9a03-517241fa8c9f-metrics-client-ca\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004225 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004234 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-web-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004241 2582 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-cluster-tls-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004250 2582 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f90bfdf-3d25-4440-9a03-517241fa8c9f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004258 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9f90bfdf-3d25-4440-9a03-517241fa8c9f-tls-assets\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.004494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.004266 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9f90bfdf-3d25-4440-9a03-517241fa8c9f-config-out\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:06:47.102858 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.102829 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 03 16:06:47.107471 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.107445 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 03 16:06:47.135324 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135297 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 03 16:06:47.135543 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135532 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="alertmanager" Aug 03 16:06:47.135584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135545 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="alertmanager" Aug 03 16:06:47.135584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135553 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="config-reloader" Aug 03 16:06:47.135584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135558 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="config-reloader" Aug 03 16:06:47.135584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135569 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a52b2559-1175-4905-b8d4-cb62d8709396" containerName="console" Aug 03 16:06:47.135584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135574 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a52b2559-1175-4905-b8d4-cb62d8709396" containerName="console" Aug 03 16:06:47.135584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135582 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy" Aug 03 16:06:47.135584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135586 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135593 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="prom-label-proxy" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135598 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="prom-label-proxy" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135611 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy-web" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135631 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy-web" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135641 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="init-config-reloader" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135647 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="init-config-reloader" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135653 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy-metric" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135658 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy-metric" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135697 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="alertmanager" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135705 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="config-reloader" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135711 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a52b2559-1175-4905-b8d4-cb62d8709396" containerName="console" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135716 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="prom-label-proxy" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135723 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy-web" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135728 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy-metric" Aug 03 16:06:47.135797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.135733 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" containerName="kube-rbac-proxy" Aug 03 16:06:47.139354 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.139338 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.142173 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.142152 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Aug 03 16:06:47.142272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.142236 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Aug 03 16:06:47.142328 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.142293 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Aug 03 16:06:47.142383 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.142371 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Aug 03 16:06:47.142454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.142434 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Aug 03 16:06:47.142666 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.142650 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-j7xt7\"" Aug 03 16:06:47.142741 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.142650 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Aug 03 16:06:47.142792 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.142655 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Aug 03 16:06:47.142792 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.142732 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Aug 03 16:06:47.147596 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.147578 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Aug 03 16:06:47.157990 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.157971 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 03 16:06:47.305952 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.305918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4860400e-2aaa-4232-b1e7-219eb243093f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.305952 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.305954 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4860400e-2aaa-4232-b1e7-219eb243093f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.305975 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306029 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4860400e-2aaa-4232-b1e7-219eb243093f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306077 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306127 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306159 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-config-volume\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306211 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306481 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm7zz\" (UniqueName: \"kubernetes.io/projected/4860400e-2aaa-4232-b1e7-219eb243093f-kube-api-access-sm7zz\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306481 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306272 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-web-config\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306481 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306300 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4860400e-2aaa-4232-b1e7-219eb243093f-config-out\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306481 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306334 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4860400e-2aaa-4232-b1e7-219eb243093f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.306481 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.306362 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407196 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407246 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm7zz\" (UniqueName: \"kubernetes.io/projected/4860400e-2aaa-4232-b1e7-219eb243093f-kube-api-access-sm7zz\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407271 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-web-config\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407290 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4860400e-2aaa-4232-b1e7-219eb243093f-config-out\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407309 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4860400e-2aaa-4232-b1e7-219eb243093f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407714 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407328 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407714 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407359 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4860400e-2aaa-4232-b1e7-219eb243093f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407714 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4860400e-2aaa-4232-b1e7-219eb243093f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407714 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407407 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407714 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407442 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4860400e-2aaa-4232-b1e7-219eb243093f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407714 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407714 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.407714 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407531 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-config-volume\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.408093 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.407846 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4860400e-2aaa-4232-b1e7-219eb243093f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.408335 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.408239 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4860400e-2aaa-4232-b1e7-219eb243093f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.410299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.410267 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4860400e-2aaa-4232-b1e7-219eb243093f-config-out\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.410460 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.410412 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-web-config\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.410460 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.410412 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.410660 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.410529 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4860400e-2aaa-4232-b1e7-219eb243093f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.410660 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.410567 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-config-volume\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.410774 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.410754 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.410964 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.410945 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.411137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.411116 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.411187 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.411174 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4860400e-2aaa-4232-b1e7-219eb243093f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.412319 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.412299 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4860400e-2aaa-4232-b1e7-219eb243093f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.415788 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.415768 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm7zz\" (UniqueName: \"kubernetes.io/projected/4860400e-2aaa-4232-b1e7-219eb243093f-kube-api-access-sm7zz\") pod \"alertmanager-main-0\" (UID: \"4860400e-2aaa-4232-b1e7-219eb243093f\") " pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.448261 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.448217 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Aug 03 16:06:47.581958 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.581931 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Aug 03 16:06:47.584268 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:06:47.584243 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4860400e_2aaa_4232_b1e7_219eb243093f.slice/crio-00838aa50cdf683f4f2e728e6dd84ba2310c8560ca2cbeba5f2ac21d8b36404e WatchSource:0}: Error finding container 00838aa50cdf683f4f2e728e6dd84ba2310c8560ca2cbeba5f2ac21d8b36404e: Status 404 returned error can't find the container with id 00838aa50cdf683f4f2e728e6dd84ba2310c8560ca2cbeba5f2ac21d8b36404e Aug 03 16:06:47.782576 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.782535 2582 generic.go:356] "Generic (PLEG): container finished" podID="4860400e-2aaa-4232-b1e7-219eb243093f" containerID="180cab855c527012011e527a8dd7062af79eabaade4b5b9d939d31eefad8a5d8" exitCode=0 Aug 03 16:06:47.782965 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.782585 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4860400e-2aaa-4232-b1e7-219eb243093f","Type":"ContainerDied","Data":"180cab855c527012011e527a8dd7062af79eabaade4b5b9d939d31eefad8a5d8"} Aug 03 16:06:47.782965 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.782658 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4860400e-2aaa-4232-b1e7-219eb243093f","Type":"ContainerStarted","Data":"00838aa50cdf683f4f2e728e6dd84ba2310c8560ca2cbeba5f2ac21d8b36404e"} Aug 03 16:06:47.998791 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:47.998766 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f90bfdf-3d25-4440-9a03-517241fa8c9f" path="/var/lib/kubelet/pods/9f90bfdf-3d25-4440-9a03-517241fa8c9f/volumes" Aug 03 16:06:48.790180 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:48.790146 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4860400e-2aaa-4232-b1e7-219eb243093f","Type":"ContainerStarted","Data":"e719f8ed648912a3a72a6c6083f67188aa3aa15c9ee7c6e4bd620568e78e0f0b"} Aug 03 16:06:48.790557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:48.790210 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4860400e-2aaa-4232-b1e7-219eb243093f","Type":"ContainerStarted","Data":"8c6a2334148da86a71e3fb5418df14fc6ee9c5cb847b011046e282ad784d1801"} Aug 03 16:06:48.790557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:48.790238 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4860400e-2aaa-4232-b1e7-219eb243093f","Type":"ContainerStarted","Data":"1c67f92f4587e12567f6742befc22c802d5a029268b1752c7c6eba8be1e664cf"} Aug 03 16:06:48.790557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:48.790263 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4860400e-2aaa-4232-b1e7-219eb243093f","Type":"ContainerStarted","Data":"67f6c0281ef1c5d1bc355f51e009126fcde4ea81932028810c45f2b41125ba18"} Aug 03 16:06:48.790557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:48.790286 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4860400e-2aaa-4232-b1e7-219eb243093f","Type":"ContainerStarted","Data":"c09cc88a0f47a1cf2f6bb22bedbdb2239b3b6c58c13f4a27f4cd58e479513c99"} Aug 03 16:06:48.790557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:48.790311 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4860400e-2aaa-4232-b1e7-219eb243093f","Type":"ContainerStarted","Data":"825efe764a1600e6838078af0ed8af3dea10bd6c24f0e3189ace9a0b8e06fa43"} Aug 03 16:06:48.819308 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:48.819225 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.80442163 podStartE2EDuration="1.819208859s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.787229ms" isStatefulPod=false podCreationTimestamp="2026-08-03 16:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:06:48.817394919 +0000 UTC m=+147.375477507" watchObservedRunningTime="2026-08-03 16:06:48.819208859 +0000 UTC m=+147.377291444" Aug 03 16:06:49.495484 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.495450 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cf644d898-7qqpr"] Aug 03 16:06:49.497871 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.497856 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.500795 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.500775 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Aug 03 16:06:49.500910 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.500775 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Aug 03 16:06:49.500910 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.500888 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hmwwx\"" Aug 03 16:06:49.501031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.500909 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Aug 03 16:06:49.501031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.500927 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Aug 03 16:06:49.501031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.501012 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Aug 03 16:06:49.506985 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.506963 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Aug 03 16:06:49.509840 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.509819 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cf644d898-7qqpr"] Aug 03 16:06:49.624812 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.624782 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-secret-telemeter-client\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.624982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.624820 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-telemeter-client-tls\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.624982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.624841 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318cc044-cddf-40e5-8285-8bf12b11949d-metrics-client-ca\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.624982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.624856 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318cc044-cddf-40e5-8285-8bf12b11949d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.624982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.624926 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318cc044-cddf-40e5-8285-8bf12b11949d-serving-certs-ca-bundle\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.624982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.624963 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-federate-client-tls\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.624982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.624981 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf2qc\" (UniqueName: \"kubernetes.io/projected/318cc044-cddf-40e5-8285-8bf12b11949d-kube-api-access-pf2qc\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.625173 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.625000 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.726121 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.726083 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-secret-telemeter-client\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.726270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.726134 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-telemeter-client-tls\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.726270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.726163 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318cc044-cddf-40e5-8285-8bf12b11949d-metrics-client-ca\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.726270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.726184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318cc044-cddf-40e5-8285-8bf12b11949d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.726455 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.726351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318cc044-cddf-40e5-8285-8bf12b11949d-serving-certs-ca-bundle\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.726455 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.726407 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-federate-client-tls\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.726455 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.726432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf2qc\" (UniqueName: \"kubernetes.io/projected/318cc044-cddf-40e5-8285-8bf12b11949d-kube-api-access-pf2qc\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.726605 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.726468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.727109 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.727038 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/318cc044-cddf-40e5-8285-8bf12b11949d-metrics-client-ca\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.727817 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.727790 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318cc044-cddf-40e5-8285-8bf12b11949d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.728127 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.728099 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/318cc044-cddf-40e5-8285-8bf12b11949d-serving-certs-ca-bundle\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.728839 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.728814 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-telemeter-client-tls\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.728942 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.728931 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-secret-telemeter-client\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.729048 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.729024 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-federate-client-tls\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.729106 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.729044 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/318cc044-cddf-40e5-8285-8bf12b11949d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.734827 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.734805 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf2qc\" (UniqueName: \"kubernetes.io/projected/318cc044-cddf-40e5-8285-8bf12b11949d-kube-api-access-pf2qc\") pod \"telemeter-client-6cf644d898-7qqpr\" (UID: \"318cc044-cddf-40e5-8285-8bf12b11949d\") " pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.808150 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.808126 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" Aug 03 16:06:49.940561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:49.940531 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cf644d898-7qqpr"] Aug 03 16:06:49.943928 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:06:49.943905 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod318cc044_cddf_40e5_8285_8bf12b11949d.slice/crio-537eabb882ad6b191f1e53093c8632777eabcf81ab8a1cb20c99db8d8eb155bc WatchSource:0}: Error finding container 537eabb882ad6b191f1e53093c8632777eabcf81ab8a1cb20c99db8d8eb155bc: Status 404 returned error can't find the container with id 537eabb882ad6b191f1e53093c8632777eabcf81ab8a1cb20c99db8d8eb155bc Aug 03 16:06:50.799454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:50.799419 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" event={"ID":"318cc044-cddf-40e5-8285-8bf12b11949d","Type":"ContainerStarted","Data":"537eabb882ad6b191f1e53093c8632777eabcf81ab8a1cb20c99db8d8eb155bc"} Aug 03 16:06:52.809321 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.809288 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" event={"ID":"318cc044-cddf-40e5-8285-8bf12b11949d","Type":"ContainerStarted","Data":"8daba1ff9c5713809ec09baa8514039b77aa814c56d956ffbe78450086dc196e"} Aug 03 16:06:52.809846 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.809354 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" event={"ID":"318cc044-cddf-40e5-8285-8bf12b11949d","Type":"ContainerStarted","Data":"df2635010656c42a9af508b3bc379afd709f6ba1ff3f6e043b167c0a61e8b633"} Aug 03 16:06:52.809846 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.809383 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" event={"ID":"318cc044-cddf-40e5-8285-8bf12b11949d","Type":"ContainerStarted","Data":"60debd7eac00205206d38f647ca8790e4d044a83524e6cd108cdda07021d584c"} Aug 03 16:06:52.834719 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.834667 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cf644d898-7qqpr" podStartSLOduration=1.6166447819999998 podStartE2EDuration="3.834653217s" totalImagesPullingTime="2.218008435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:06:52.832684632 +0000 UTC m=+151.390767230" watchObservedRunningTime="2026-08-03 16:06:52.834653217 +0000 UTC m=+151.392735803" Aug 03 16:06:52.917246 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.917212 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7479955746-hqrrd"] Aug 03 16:06:52.919406 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.919385 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:52.931109 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.931082 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7479955746-hqrrd"] Aug 03 16:06:52.952487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.952461 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-oauth-serving-cert\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:52.952611 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.952502 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-oauth-config\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:52.952611 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.952525 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-serving-cert\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:52.952611 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.952544 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-trusted-ca-bundle\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:52.952800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.952609 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q9tv\" (UniqueName: \"kubernetes.io/projected/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-kube-api-access-5q9tv\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:52.952800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.952674 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-config\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:52.952800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:52.952715 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-service-ca\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.053843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.053804 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-oauth-serving-cert\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.053843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.053847 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-oauth-config\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.054097 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.053867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-serving-cert\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.054097 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.053886 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-trusted-ca-bundle\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.054097 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.053915 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q9tv\" (UniqueName: \"kubernetes.io/projected/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-kube-api-access-5q9tv\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.054097 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.053939 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-config\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.054097 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.053968 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-service-ca\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.054730 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.054708 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-oauth-serving-cert\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.054836 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.054811 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-config\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.054877 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.054838 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-service-ca\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.054917 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.054901 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-trusted-ca-bundle\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.056250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.056229 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-oauth-config\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.056466 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.056431 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-serving-cert\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.063195 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.063150 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q9tv\" (UniqueName: \"kubernetes.io/projected/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-kube-api-access-5q9tv\") pod \"console-7479955746-hqrrd\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.227530 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.227483 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:06:53.354024 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.354000 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7479955746-hqrrd"] Aug 03 16:06:53.356430 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:06:53.356402 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9469cdb0_4ee4_496b_8a2b_a7b3b94871d1.slice/crio-3e890cb56460872ed6a599ba734e53dbcabce418e2d488dbb0a65780e9b12565 WatchSource:0}: Error finding container 3e890cb56460872ed6a599ba734e53dbcabce418e2d488dbb0a65780e9b12565: Status 404 returned error can't find the container with id 3e890cb56460872ed6a599ba734e53dbcabce418e2d488dbb0a65780e9b12565 Aug 03 16:06:53.814524 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.814482 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7479955746-hqrrd" event={"ID":"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1","Type":"ContainerStarted","Data":"c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb"} Aug 03 16:06:53.814920 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.814534 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7479955746-hqrrd" event={"ID":"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1","Type":"ContainerStarted","Data":"3e890cb56460872ed6a599ba734e53dbcabce418e2d488dbb0a65780e9b12565"} Aug 03 16:06:53.833322 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:06:53.833273 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7479955746-hqrrd" podStartSLOduration=1.833256412 podStartE2EDuration="1.833256412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:06:53.832642282 +0000 UTC m=+152.390724869" watchObservedRunningTime="2026-08-03 16:06:53.833256412 +0000 UTC m=+152.391338997" Aug 03 16:07:03.227896 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:03.227851 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:07:03.228318 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:03.227916 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:07:03.232768 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:03.232747 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:07:03.856169 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:03.856138 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:07:03.904632 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:03.904588 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74d6b7b77c-5mw9s"] Aug 03 16:07:28.924041 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:28.923994 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74d6b7b77c-5mw9s" podUID="26e7da64-796c-4475-aa2a-f54d3a7a9ef8" containerName="console" containerID="cri-o://feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7" gracePeriod=15 Aug 03 16:07:29.153733 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.153704 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:07:29.238303 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.238212 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-trusted-ca-bundle\") pod \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " Aug 03 16:07:29.238303 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.238271 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-oauth-serving-cert\") pod \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " Aug 03 16:07:29.238303 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.238300 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-config\") pod \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " Aug 03 16:07:29.238566 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.238342 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f92h9\" (UniqueName: \"kubernetes.io/projected/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-kube-api-access-f92h9\") pod \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " Aug 03 16:07:29.238566 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.238369 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-oauth-config\") pod \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " Aug 03 16:07:29.238566 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.238394 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-serving-cert\") pod \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " Aug 03 16:07:29.238566 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.238433 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-service-ca\") pod \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\" (UID: \"26e7da64-796c-4475-aa2a-f54d3a7a9ef8\") " Aug 03 16:07:29.238930 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.238902 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "26e7da64-796c-4475-aa2a-f54d3a7a9ef8" (UID: "26e7da64-796c-4475-aa2a-f54d3a7a9ef8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:07:29.239026 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.238916 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-config" (OuterVolumeSpecName: "console-config") pod "26e7da64-796c-4475-aa2a-f54d3a7a9ef8" (UID: "26e7da64-796c-4475-aa2a-f54d3a7a9ef8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:07:29.239074 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.239032 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "26e7da64-796c-4475-aa2a-f54d3a7a9ef8" (UID: "26e7da64-796c-4475-aa2a-f54d3a7a9ef8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:07:29.239074 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.239048 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-service-ca" (OuterVolumeSpecName: "service-ca") pod "26e7da64-796c-4475-aa2a-f54d3a7a9ef8" (UID: "26e7da64-796c-4475-aa2a-f54d3a7a9ef8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:07:29.240675 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.240651 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-kube-api-access-f92h9" (OuterVolumeSpecName: "kube-api-access-f92h9") pod "26e7da64-796c-4475-aa2a-f54d3a7a9ef8" (UID: "26e7da64-796c-4475-aa2a-f54d3a7a9ef8"). InnerVolumeSpecName "kube-api-access-f92h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:07:29.240675 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.240651 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "26e7da64-796c-4475-aa2a-f54d3a7a9ef8" (UID: "26e7da64-796c-4475-aa2a-f54d3a7a9ef8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:07:29.240823 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.240737 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "26e7da64-796c-4475-aa2a-f54d3a7a9ef8" (UID: "26e7da64-796c-4475-aa2a-f54d3a7a9ef8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:07:29.339385 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.339347 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-oauth-serving-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:07:29.339385 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.339378 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:07:29.339385 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.339390 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f92h9\" (UniqueName: \"kubernetes.io/projected/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-kube-api-access-f92h9\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:07:29.339609 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.339403 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-oauth-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:07:29.339609 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.339418 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-console-serving-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:07:29.339609 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.339429 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-service-ca\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:07:29.339609 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.339439 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26e7da64-796c-4475-aa2a-f54d3a7a9ef8-trusted-ca-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:07:29.954060 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.954022 2582 generic.go:356] "Generic (PLEG): container finished" podID="26e7da64-796c-4475-aa2a-f54d3a7a9ef8" containerID="feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7" exitCode=0 Aug 03 16:07:29.954442 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.954070 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74d6b7b77c-5mw9s" Aug 03 16:07:29.954442 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.954089 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d6b7b77c-5mw9s" event={"ID":"26e7da64-796c-4475-aa2a-f54d3a7a9ef8","Type":"ContainerDied","Data":"feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7"} Aug 03 16:07:29.954442 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.954137 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74d6b7b77c-5mw9s" event={"ID":"26e7da64-796c-4475-aa2a-f54d3a7a9ef8","Type":"ContainerDied","Data":"2224638636f0d9693c5abf2adba67590f3cbbbeb3f69c3a61bcb6ae5d1f24371"} Aug 03 16:07:29.954442 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.954166 2582 scope.go:117] "RemoveContainer" containerID="feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7" Aug 03 16:07:29.962394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.962370 2582 scope.go:117] "RemoveContainer" containerID="feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7" Aug 03 16:07:29.962657 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:07:29.962632 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7\": container with ID starting with feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7 not found: ID does not exist" containerID="feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7" Aug 03 16:07:29.962765 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.962662 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7"} err="failed to get container status \"feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7\": rpc error: code = NotFound desc = could not find container \"feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7\": container with ID starting with feab0865fe5367b85e47dc14fdc0d9565e11f5ec7db8ef7ed572c4519c1becf7 not found: ID does not exist" Aug 03 16:07:29.975718 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.975696 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74d6b7b77c-5mw9s"] Aug 03 16:07:29.979951 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.979930 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74d6b7b77c-5mw9s"] Aug 03 16:07:29.993031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:07:29.993004 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26e7da64-796c-4475-aa2a-f54d3a7a9ef8" path="/var/lib/kubelet/pods/26e7da64-796c-4475-aa2a-f54d3a7a9ef8/volumes" Aug 03 16:08:05.006375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.006343 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d558fbc7f-j2h6x"] Aug 03 16:08:05.006858 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.006662 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e7da64-796c-4475-aa2a-f54d3a7a9ef8" containerName="console" Aug 03 16:08:05.006858 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.006673 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e7da64-796c-4475-aa2a-f54d3a7a9ef8" containerName="console" Aug 03 16:08:05.006858 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.006728 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="26e7da64-796c-4475-aa2a-f54d3a7a9ef8" containerName="console" Aug 03 16:08:05.009224 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.009205 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.020214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.020192 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d558fbc7f-j2h6x"] Aug 03 16:08:05.029955 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.029931 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8g78\" (UniqueName: \"kubernetes.io/projected/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-kube-api-access-f8g78\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.030057 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.029962 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-service-ca\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.030057 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.029982 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-trusted-ca-bundle\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.030057 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.030006 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-oauth-serving-cert\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.030258 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.030088 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-config\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.030258 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.030155 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-serving-cert\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.030258 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.030187 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-oauth-config\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.131056 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131024 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-serving-cert\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.131056 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131056 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-oauth-config\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.131289 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131084 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8g78\" (UniqueName: \"kubernetes.io/projected/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-kube-api-access-f8g78\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.131289 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131102 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-service-ca\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.131289 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-trusted-ca-bundle\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.131289 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131169 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-oauth-serving-cert\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.131289 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131204 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-config\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.131955 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131928 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-service-ca\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.132060 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131980 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-config\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.132060 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.131985 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-oauth-serving-cert\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.132060 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.132046 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-trusted-ca-bundle\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.133611 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.133593 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-oauth-config\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.133684 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.133666 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-serving-cert\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.141484 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.141466 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8g78\" (UniqueName: \"kubernetes.io/projected/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-kube-api-access-f8g78\") pod \"console-5d558fbc7f-j2h6x\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.318180 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.318142 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:05.445661 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:05.445635 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d558fbc7f-j2h6x"] Aug 03 16:08:05.449343 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:08:05.449316 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c5a66a_ec47_4dfe_9a50_7c3190662a1d.slice/crio-d9d9dc227ef3b622be32193e2a717ea8fe18aaa2139e08b9cabf3df7afbabecd WatchSource:0}: Error finding container d9d9dc227ef3b622be32193e2a717ea8fe18aaa2139e08b9cabf3df7afbabecd: Status 404 returned error can't find the container with id d9d9dc227ef3b622be32193e2a717ea8fe18aaa2139e08b9cabf3df7afbabecd Aug 03 16:08:06.080065 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:06.080025 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d558fbc7f-j2h6x" event={"ID":"78c5a66a-ec47-4dfe-9a50-7c3190662a1d","Type":"ContainerStarted","Data":"a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a"} Aug 03 16:08:06.080434 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:06.080083 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d558fbc7f-j2h6x" event={"ID":"78c5a66a-ec47-4dfe-9a50-7c3190662a1d","Type":"ContainerStarted","Data":"d9d9dc227ef3b622be32193e2a717ea8fe18aaa2139e08b9cabf3df7afbabecd"} Aug 03 16:08:06.102268 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:06.102222 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d558fbc7f-j2h6x" podStartSLOduration=2.102208048 podStartE2EDuration="2.102208048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:08:06.100477189 +0000 UTC m=+224.658559766" watchObservedRunningTime="2026-08-03 16:08:06.102208048 +0000 UTC m=+224.660290634" Aug 03 16:08:15.318461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:15.318424 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:15.318878 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:15.318485 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:15.323092 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:15.323071 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:16.124382 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:16.124356 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:08:16.179309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:16.179282 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7479955746-hqrrd"] Aug 03 16:08:41.198522 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.198448 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7479955746-hqrrd" podUID="9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" containerName="console" containerID="cri-o://c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb" gracePeriod=15 Aug 03 16:08:41.428774 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.428752 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:08:41.434862 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.434843 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-oauth-serving-cert\") pod \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " Aug 03 16:08:41.434946 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.434871 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-config\") pod \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " Aug 03 16:08:41.434946 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.434891 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q9tv\" (UniqueName: \"kubernetes.io/projected/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-kube-api-access-5q9tv\") pod \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " Aug 03 16:08:41.434946 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.434915 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-trusted-ca-bundle\") pod \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " Aug 03 16:08:41.435050 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.434974 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-oauth-config\") pod \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " Aug 03 16:08:41.435050 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.435003 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-serving-cert\") pod \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " Aug 03 16:08:41.435125 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.435048 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-service-ca\") pod \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\" (UID: \"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1\") " Aug 03 16:08:41.435256 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.435219 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" (UID: "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:08:41.435468 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.435443 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-oauth-serving-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:08:41.435571 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.435464 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" (UID: "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:08:41.435571 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.435513 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-service-ca" (OuterVolumeSpecName: "service-ca") pod "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" (UID: "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:08:41.435571 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.435511 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-config" (OuterVolumeSpecName: "console-config") pod "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" (UID: "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:08:41.436903 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.436879 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" (UID: "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:08:41.437041 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.437026 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-kube-api-access-5q9tv" (OuterVolumeSpecName: "kube-api-access-5q9tv") pod "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" (UID: "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1"). InnerVolumeSpecName "kube-api-access-5q9tv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:08:41.437082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.437049 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" (UID: "9469cdb0-4ee4-496b-8a2b-a7b3b94871d1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:08:41.536134 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.536109 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-oauth-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:08:41.536134 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.536135 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-serving-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:08:41.536318 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.536145 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-service-ca\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:08:41.536318 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.536155 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-console-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:08:41.536318 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.536163 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q9tv\" (UniqueName: \"kubernetes.io/projected/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-kube-api-access-5q9tv\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:08:41.536318 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:41.536171 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1-trusted-ca-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:08:42.218149 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:42.218118 2582 generic.go:356] "Generic (PLEG): container finished" podID="9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" containerID="c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb" exitCode=0 Aug 03 16:08:42.218574 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:42.218161 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7479955746-hqrrd" Aug 03 16:08:42.218574 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:42.218168 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7479955746-hqrrd" event={"ID":"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1","Type":"ContainerDied","Data":"c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb"} Aug 03 16:08:42.218574 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:42.218209 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7479955746-hqrrd" event={"ID":"9469cdb0-4ee4-496b-8a2b-a7b3b94871d1","Type":"ContainerDied","Data":"3e890cb56460872ed6a599ba734e53dbcabce418e2d488dbb0a65780e9b12565"} Aug 03 16:08:42.218574 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:42.218230 2582 scope.go:117] "RemoveContainer" containerID="c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb" Aug 03 16:08:42.230230 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:42.230211 2582 scope.go:117] "RemoveContainer" containerID="c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb" Aug 03 16:08:42.230508 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:08:42.230485 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb\": container with ID starting with c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb not found: ID does not exist" containerID="c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb" Aug 03 16:08:42.230605 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:42.230511 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb"} err="failed to get container status \"c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb\": rpc error: code = NotFound desc = could not find container \"c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb\": container with ID starting with c083e705297aad333162e429c6cc4d4bf192835b7b86c92412d48394c9dda1cb not found: ID does not exist" Aug 03 16:08:42.239749 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:42.239729 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7479955746-hqrrd"] Aug 03 16:08:42.241792 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:42.241775 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7479955746-hqrrd"] Aug 03 16:08:43.993424 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:08:43.993352 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" path="/var/lib/kubelet/pods/9469cdb0-4ee4-496b-8a2b-a7b3b94871d1/volumes" Aug 03 16:09:03.443451 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.443418 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7"] Aug 03 16:09:03.443835 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.443743 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" containerName="console" Aug 03 16:09:03.443835 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.443754 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" containerName="console" Aug 03 16:09:03.443835 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.443799 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9469cdb0-4ee4-496b-8a2b-a7b3b94871d1" containerName="console" Aug 03 16:09:03.446005 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.445991 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.448691 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.448670 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 03 16:09:03.450051 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.450035 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fqmmj\"" Aug 03 16:09:03.450113 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.450051 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 03 16:09:03.455432 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.455411 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7"] Aug 03 16:09:03.492547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.492516 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.492699 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.492554 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.492699 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.492579 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfjrx\" (UniqueName: \"kubernetes.io/projected/42c29582-952e-4651-b940-e1850fe27f5b-kube-api-access-xfjrx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.593875 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.593841 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.594042 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.593883 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.594042 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.593918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfjrx\" (UniqueName: \"kubernetes.io/projected/42c29582-952e-4651-b940-e1850fe27f5b-kube-api-access-xfjrx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.594253 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.594234 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.594291 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.594262 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.604179 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.604148 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfjrx\" (UniqueName: \"kubernetes.io/projected/42c29582-952e-4651-b940-e1850fe27f5b-kube-api-access-xfjrx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.754824 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.754725 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:03.881377 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:03.881356 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7"] Aug 03 16:09:03.883647 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:09:03.883604 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c29582_952e_4651_b940_e1850fe27f5b.slice/crio-fab4ba474932474bb1debf0c47edb20f86af81c97c496b78618fddab55d19ba9 WatchSource:0}: Error finding container fab4ba474932474bb1debf0c47edb20f86af81c97c496b78618fddab55d19ba9: Status 404 returned error can't find the container with id fab4ba474932474bb1debf0c47edb20f86af81c97c496b78618fddab55d19ba9 Aug 03 16:09:04.297702 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:04.297670 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" event={"ID":"42c29582-952e-4651-b940-e1850fe27f5b","Type":"ContainerStarted","Data":"fab4ba474932474bb1debf0c47edb20f86af81c97c496b78618fddab55d19ba9"} Aug 03 16:09:12.329855 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:12.329816 2582 generic.go:356] "Generic (PLEG): container finished" podID="42c29582-952e-4651-b940-e1850fe27f5b" containerID="88c4c0857d059da099afb1ee4aeeae30bd6f74a9e56de22249fea045c1b12d84" exitCode=0 Aug 03 16:09:12.330421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:12.329866 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" event={"ID":"42c29582-952e-4651-b940-e1850fe27f5b","Type":"ContainerDied","Data":"88c4c0857d059da099afb1ee4aeeae30bd6f74a9e56de22249fea045c1b12d84"} Aug 03 16:09:15.342108 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:15.342074 2582 generic.go:356] "Generic (PLEG): container finished" podID="42c29582-952e-4651-b940-e1850fe27f5b" containerID="6b334b490ef5cad26b37c268fdfbb5a93c499e72ee6be9ce649dedc1f6ea4618" exitCode=0 Aug 03 16:09:15.342470 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:15.342126 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" event={"ID":"42c29582-952e-4651-b940-e1850fe27f5b","Type":"ContainerDied","Data":"6b334b490ef5cad26b37c268fdfbb5a93c499e72ee6be9ce649dedc1f6ea4618"} Aug 03 16:09:22.428779 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:22.428756 2582 kubelet.go:1648] "Image garbage collection succeeded" Aug 03 16:09:23.377758 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:23.377722 2582 generic.go:356] "Generic (PLEG): container finished" podID="42c29582-952e-4651-b940-e1850fe27f5b" containerID="6849c66a56ab667aeaf105e2478c66550fc0fe778d4ab713717445bab5868809" exitCode=0 Aug 03 16:09:23.377929 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:23.377811 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" event={"ID":"42c29582-952e-4651-b940-e1850fe27f5b","Type":"ContainerDied","Data":"6849c66a56ab667aeaf105e2478c66550fc0fe778d4ab713717445bab5868809"} Aug 03 16:09:24.529062 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.529032 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:24.673813 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.673724 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfjrx\" (UniqueName: \"kubernetes.io/projected/42c29582-952e-4651-b940-e1850fe27f5b-kube-api-access-xfjrx\") pod \"42c29582-952e-4651-b940-e1850fe27f5b\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " Aug 03 16:09:24.673813 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.673773 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-util\") pod \"42c29582-952e-4651-b940-e1850fe27f5b\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " Aug 03 16:09:24.673813 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.673807 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-bundle\") pod \"42c29582-952e-4651-b940-e1850fe27f5b\" (UID: \"42c29582-952e-4651-b940-e1850fe27f5b\") " Aug 03 16:09:24.674484 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.674454 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-bundle" (OuterVolumeSpecName: "bundle") pod "42c29582-952e-4651-b940-e1850fe27f5b" (UID: "42c29582-952e-4651-b940-e1850fe27f5b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:09:24.675871 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.675845 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42c29582-952e-4651-b940-e1850fe27f5b-kube-api-access-xfjrx" (OuterVolumeSpecName: "kube-api-access-xfjrx") pod "42c29582-952e-4651-b940-e1850fe27f5b" (UID: "42c29582-952e-4651-b940-e1850fe27f5b"). InnerVolumeSpecName "kube-api-access-xfjrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:09:24.677600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.677573 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-util" (OuterVolumeSpecName: "util") pod "42c29582-952e-4651-b940-e1850fe27f5b" (UID: "42c29582-952e-4651-b940-e1850fe27f5b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:09:24.774547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.774510 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfjrx\" (UniqueName: \"kubernetes.io/projected/42c29582-952e-4651-b940-e1850fe27f5b-kube-api-access-xfjrx\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:09:24.774547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.774540 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-util\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:09:24.774547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:24.774550 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42c29582-952e-4651-b940-e1850fe27f5b-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:09:25.386844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:25.386817 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" Aug 03 16:09:25.387001 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:25.386815 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wwgm7" event={"ID":"42c29582-952e-4651-b940-e1850fe27f5b","Type":"ContainerDied","Data":"fab4ba474932474bb1debf0c47edb20f86af81c97c496b78618fddab55d19ba9"} Aug 03 16:09:25.387001 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:25.386937 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fab4ba474932474bb1debf0c47edb20f86af81c97c496b78618fddab55d19ba9" Aug 03 16:09:31.480770 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.480739 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp"] Aug 03 16:09:31.481129 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.481039 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42c29582-952e-4651-b940-e1850fe27f5b" containerName="util" Aug 03 16:09:31.481129 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.481050 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="42c29582-952e-4651-b940-e1850fe27f5b" containerName="util" Aug 03 16:09:31.481129 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.481071 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42c29582-952e-4651-b940-e1850fe27f5b" containerName="pull" Aug 03 16:09:31.481129 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.481076 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="42c29582-952e-4651-b940-e1850fe27f5b" containerName="pull" Aug 03 16:09:31.481129 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.481084 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42c29582-952e-4651-b940-e1850fe27f5b" containerName="extract" Aug 03 16:09:31.481129 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.481089 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="42c29582-952e-4651-b940-e1850fe27f5b" containerName="extract" Aug 03 16:09:31.481129 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.481130 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="42c29582-952e-4651-b940-e1850fe27f5b" containerName="extract" Aug 03 16:09:31.483076 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.483062 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" Aug 03 16:09:31.490841 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.490816 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Aug 03 16:09:31.490963 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.490946 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Aug 03 16:09:31.492386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.492369 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-jzq8n\"" Aug 03 16:09:31.505739 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.505717 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp"] Aug 03 16:09:31.631585 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.631548 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpk6z\" (UniqueName: \"kubernetes.io/projected/0d0d8021-9464-4510-a4b7-aee262222e4e-kube-api-access-bpk6z\") pod \"cert-manager-operator-controller-manager-55c444bf86-stdvp\" (UID: \"0d0d8021-9464-4510-a4b7-aee262222e4e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" Aug 03 16:09:31.631801 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.631682 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d0d8021-9464-4510-a4b7-aee262222e4e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-stdvp\" (UID: \"0d0d8021-9464-4510-a4b7-aee262222e4e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" Aug 03 16:09:31.732969 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.732880 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk6z\" (UniqueName: \"kubernetes.io/projected/0d0d8021-9464-4510-a4b7-aee262222e4e-kube-api-access-bpk6z\") pod \"cert-manager-operator-controller-manager-55c444bf86-stdvp\" (UID: \"0d0d8021-9464-4510-a4b7-aee262222e4e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" Aug 03 16:09:31.733114 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.733027 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d0d8021-9464-4510-a4b7-aee262222e4e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-stdvp\" (UID: \"0d0d8021-9464-4510-a4b7-aee262222e4e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" Aug 03 16:09:31.733355 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.733338 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d0d8021-9464-4510-a4b7-aee262222e4e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-stdvp\" (UID: \"0d0d8021-9464-4510-a4b7-aee262222e4e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" Aug 03 16:09:31.742287 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.742258 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpk6z\" (UniqueName: \"kubernetes.io/projected/0d0d8021-9464-4510-a4b7-aee262222e4e-kube-api-access-bpk6z\") pod \"cert-manager-operator-controller-manager-55c444bf86-stdvp\" (UID: \"0d0d8021-9464-4510-a4b7-aee262222e4e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" Aug 03 16:09:31.791385 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.791351 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" Aug 03 16:09:31.931770 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.931750 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp"] Aug 03 16:09:31.934136 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:09:31.934110 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0d8021_9464_4510_a4b7_aee262222e4e.slice/crio-3b5f307457729cee45bbd0d9cad42ed6f935495767faf1c5d762ab5bbe4551fa WatchSource:0}: Error finding container 3b5f307457729cee45bbd0d9cad42ed6f935495767faf1c5d762ab5bbe4551fa: Status 404 returned error can't find the container with id 3b5f307457729cee45bbd0d9cad42ed6f935495767faf1c5d762ab5bbe4551fa Aug 03 16:09:31.936784 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:31.936767 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 03 16:09:32.413826 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:32.413793 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" event={"ID":"0d0d8021-9464-4510-a4b7-aee262222e4e","Type":"ContainerStarted","Data":"3b5f307457729cee45bbd0d9cad42ed6f935495767faf1c5d762ab5bbe4551fa"} Aug 03 16:09:34.424599 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:34.424562 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" event={"ID":"0d0d8021-9464-4510-a4b7-aee262222e4e","Type":"ContainerStarted","Data":"ac9bc01dc141a4c7a56d8df75ef02357fcacf6b9bef1319d15750c5354b9801e"} Aug 03 16:09:34.451485 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:34.451428 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-stdvp" podStartSLOduration=1.107245169 podStartE2EDuration="3.451407224s" totalImagesPullingTime="2.344162055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:09:34.449199826 +0000 UTC m=+313.007282411" watchObservedRunningTime="2026-08-03 16:09:34.451407224 +0000 UTC m=+313.009489815" Aug 03 16:09:35.527866 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.527832 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c"] Aug 03 16:09:35.530454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.530433 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.533683 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.533656 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fqmmj\"" Aug 03 16:09:35.533683 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.533669 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 03 16:09:35.533861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.533698 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 03 16:09:35.539981 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.539962 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c"] Aug 03 16:09:35.668224 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.668189 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp74f\" (UniqueName: \"kubernetes.io/projected/f71d8c7b-94ce-4f89-9826-c46468ef7498-kube-api-access-kp74f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.668390 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.668235 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.668390 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.668319 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.768851 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.768809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp74f\" (UniqueName: \"kubernetes.io/projected/f71d8c7b-94ce-4f89-9826-c46468ef7498-kube-api-access-kp74f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.769021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.768865 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.769021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.768913 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.769303 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.769283 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.769338 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.769283 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.781592 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.781526 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp74f\" (UniqueName: \"kubernetes.io/projected/f71d8c7b-94ce-4f89-9826-c46468ef7498-kube-api-access-kp74f\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.840441 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.840413 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:35.945021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.944993 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4h6lb"] Aug 03 16:09:35.947683 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.947664 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:35.950885 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.950867 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Aug 03 16:09:35.951031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.950867 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-8sv4q\"" Aug 03 16:09:35.951031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.950870 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Aug 03 16:09:35.957922 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.957902 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4h6lb"] Aug 03 16:09:35.965440 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:35.965421 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c"] Aug 03 16:09:35.968224 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:09:35.968203 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf71d8c7b_94ce_4f89_9826_c46468ef7498.slice/crio-4a6733f0114c7a378e4dd9b10178997602318aa9f62c334d25203f5ce1c2d35d WatchSource:0}: Error finding container 4a6733f0114c7a378e4dd9b10178997602318aa9f62c334d25203f5ce1c2d35d: Status 404 returned error can't find the container with id 4a6733f0114c7a378e4dd9b10178997602318aa9f62c334d25203f5ce1c2d35d Aug 03 16:09:36.071314 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.071292 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13e809bf-84af-479d-93ca-7931fb1687e2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4h6lb\" (UID: \"13e809bf-84af-479d-93ca-7931fb1687e2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:36.071412 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.071349 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm4hn\" (UniqueName: \"kubernetes.io/projected/13e809bf-84af-479d-93ca-7931fb1687e2-kube-api-access-dm4hn\") pod \"cert-manager-webhook-6486ffd997-4h6lb\" (UID: \"13e809bf-84af-479d-93ca-7931fb1687e2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:36.171945 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.171908 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13e809bf-84af-479d-93ca-7931fb1687e2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4h6lb\" (UID: \"13e809bf-84af-479d-93ca-7931fb1687e2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:36.172125 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.171991 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm4hn\" (UniqueName: \"kubernetes.io/projected/13e809bf-84af-479d-93ca-7931fb1687e2-kube-api-access-dm4hn\") pod \"cert-manager-webhook-6486ffd997-4h6lb\" (UID: \"13e809bf-84af-479d-93ca-7931fb1687e2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:36.181333 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.181313 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13e809bf-84af-479d-93ca-7931fb1687e2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4h6lb\" (UID: \"13e809bf-84af-479d-93ca-7931fb1687e2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:36.181441 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.181425 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm4hn\" (UniqueName: \"kubernetes.io/projected/13e809bf-84af-479d-93ca-7931fb1687e2-kube-api-access-dm4hn\") pod \"cert-manager-webhook-6486ffd997-4h6lb\" (UID: \"13e809bf-84af-479d-93ca-7931fb1687e2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:36.268596 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.268558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:36.389190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.389169 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4h6lb"] Aug 03 16:09:36.391174 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:09:36.391150 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13e809bf_84af_479d_93ca_7931fb1687e2.slice/crio-43e24c5c4a6b64fb60a77c3737f779b3586fa64244fdd6d1c8229c1d11d02493 WatchSource:0}: Error finding container 43e24c5c4a6b64fb60a77c3737f779b3586fa64244fdd6d1c8229c1d11d02493: Status 404 returned error can't find the container with id 43e24c5c4a6b64fb60a77c3737f779b3586fa64244fdd6d1c8229c1d11d02493 Aug 03 16:09:36.437147 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.437115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" event={"ID":"13e809bf-84af-479d-93ca-7931fb1687e2","Type":"ContainerStarted","Data":"43e24c5c4a6b64fb60a77c3737f779b3586fa64244fdd6d1c8229c1d11d02493"} Aug 03 16:09:36.438370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.438342 2582 generic.go:356] "Generic (PLEG): container finished" podID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerID="468aaf88b70e0c3daf58c13b758e6a4a6fa7e545b319730a9d3c20c2eb1d3f2a" exitCode=0 Aug 03 16:09:36.438370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.438373 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" event={"ID":"f71d8c7b-94ce-4f89-9826-c46468ef7498","Type":"ContainerDied","Data":"468aaf88b70e0c3daf58c13b758e6a4a6fa7e545b319730a9d3c20c2eb1d3f2a"} Aug 03 16:09:36.438550 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:36.438403 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" event={"ID":"f71d8c7b-94ce-4f89-9826-c46468ef7498","Type":"ContainerStarted","Data":"4a6733f0114c7a378e4dd9b10178997602318aa9f62c334d25203f5ce1c2d35d"} Aug 03 16:09:37.575535 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.571901 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jh89b"] Aug 03 16:09:37.576051 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.575687 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" Aug 03 16:09:37.579022 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.578999 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-r65dc\"" Aug 03 16:09:37.582566 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.582542 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jh89b"] Aug 03 16:09:37.684309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.684270 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f9gg\" (UniqueName: \"kubernetes.io/projected/99f9f2bf-2628-499f-bb6a-945521d615bc-kube-api-access-6f9gg\") pod \"cert-manager-cainjector-559cdd8c87-jh89b\" (UID: \"99f9f2bf-2628-499f-bb6a-945521d615bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" Aug 03 16:09:37.684485 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.684317 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99f9f2bf-2628-499f-bb6a-945521d615bc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jh89b\" (UID: \"99f9f2bf-2628-499f-bb6a-945521d615bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" Aug 03 16:09:37.785788 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.785748 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99f9f2bf-2628-499f-bb6a-945521d615bc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jh89b\" (UID: \"99f9f2bf-2628-499f-bb6a-945521d615bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" Aug 03 16:09:37.786006 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.785891 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f9gg\" (UniqueName: \"kubernetes.io/projected/99f9f2bf-2628-499f-bb6a-945521d615bc-kube-api-access-6f9gg\") pod \"cert-manager-cainjector-559cdd8c87-jh89b\" (UID: \"99f9f2bf-2628-499f-bb6a-945521d615bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" Aug 03 16:09:37.795515 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.795449 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f9gg\" (UniqueName: \"kubernetes.io/projected/99f9f2bf-2628-499f-bb6a-945521d615bc-kube-api-access-6f9gg\") pod \"cert-manager-cainjector-559cdd8c87-jh89b\" (UID: \"99f9f2bf-2628-499f-bb6a-945521d615bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" Aug 03 16:09:37.795679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.795602 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99f9f2bf-2628-499f-bb6a-945521d615bc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jh89b\" (UID: \"99f9f2bf-2628-499f-bb6a-945521d615bc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" Aug 03 16:09:37.889372 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:37.889292 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" Aug 03 16:09:39.538538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:39.538508 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jh89b"] Aug 03 16:09:39.540531 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:09:39.540497 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99f9f2bf_2628_499f_bb6a_945521d615bc.slice/crio-19bd7362dd8a95338c066c6a66b75e34a4babb9a9415e977afb6d61486aaa1f3 WatchSource:0}: Error finding container 19bd7362dd8a95338c066c6a66b75e34a4babb9a9415e977afb6d61486aaa1f3: Status 404 returned error can't find the container with id 19bd7362dd8a95338c066c6a66b75e34a4babb9a9415e977afb6d61486aaa1f3 Aug 03 16:09:40.458034 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:40.457997 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" event={"ID":"13e809bf-84af-479d-93ca-7931fb1687e2","Type":"ContainerStarted","Data":"3e40084f24b43a8b97de8deec7a76e3aa2a60933f1cd68d5893d4faea9f030f7"} Aug 03 16:09:40.458213 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:40.458052 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:40.461926 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:40.461892 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" event={"ID":"99f9f2bf-2628-499f-bb6a-945521d615bc","Type":"ContainerStarted","Data":"2c626c6bbad38250fdad62863afa8bffce305836a11cb38f3c6641df24752e6f"} Aug 03 16:09:40.462069 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:40.461934 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" event={"ID":"99f9f2bf-2628-499f-bb6a-945521d615bc","Type":"ContainerStarted","Data":"19bd7362dd8a95338c066c6a66b75e34a4babb9a9415e977afb6d61486aaa1f3"} Aug 03 16:09:40.463491 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:40.463466 2582 generic.go:356] "Generic (PLEG): container finished" podID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerID="2eafc08421b9b89fcfb9fa51af6bb72868b321cd6b920451414510b072605527" exitCode=0 Aug 03 16:09:40.463589 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:40.463499 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" event={"ID":"f71d8c7b-94ce-4f89-9826-c46468ef7498","Type":"ContainerDied","Data":"2eafc08421b9b89fcfb9fa51af6bb72868b321cd6b920451414510b072605527"} Aug 03 16:09:40.480542 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:40.480501 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" podStartSLOduration=2.475037394 podStartE2EDuration="5.480486745s" totalImagesPullingTime="3.005449351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:09:40.478655677 +0000 UTC m=+319.036738261" watchObservedRunningTime="2026-08-03 16:09:40.480486745 +0000 UTC m=+319.038569329" Aug 03 16:09:40.507750 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:40.507706 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jh89b" podStartSLOduration=3.5076922980000003 podStartE2EDuration="3.507692298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:09:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:09:40.506606806 +0000 UTC m=+319.064689391" watchObservedRunningTime="2026-08-03 16:09:40.507692298 +0000 UTC m=+319.065774882" Aug 03 16:09:41.469425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:41.469389 2582 generic.go:356] "Generic (PLEG): container finished" podID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerID="4bb4d1efdc040f2e181efb0304d844cdcac3af7e876bb740cfec64b2d2272a00" exitCode=0 Aug 03 16:09:41.469841 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:41.469481 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" event={"ID":"f71d8c7b-94ce-4f89-9826-c46468ef7498","Type":"ContainerDied","Data":"4bb4d1efdc040f2e181efb0304d844cdcac3af7e876bb740cfec64b2d2272a00"} Aug 03 16:09:42.597681 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.597648 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:42.729436 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.729354 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-bundle\") pod \"f71d8c7b-94ce-4f89-9826-c46468ef7498\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " Aug 03 16:09:42.729436 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.729393 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp74f\" (UniqueName: \"kubernetes.io/projected/f71d8c7b-94ce-4f89-9826-c46468ef7498-kube-api-access-kp74f\") pod \"f71d8c7b-94ce-4f89-9826-c46468ef7498\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " Aug 03 16:09:42.729436 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.729420 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-util\") pod \"f71d8c7b-94ce-4f89-9826-c46468ef7498\" (UID: \"f71d8c7b-94ce-4f89-9826-c46468ef7498\") " Aug 03 16:09:42.729852 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.729828 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-bundle" (OuterVolumeSpecName: "bundle") pod "f71d8c7b-94ce-4f89-9826-c46468ef7498" (UID: "f71d8c7b-94ce-4f89-9826-c46468ef7498"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:09:42.731503 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.731477 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f71d8c7b-94ce-4f89-9826-c46468ef7498-kube-api-access-kp74f" (OuterVolumeSpecName: "kube-api-access-kp74f") pod "f71d8c7b-94ce-4f89-9826-c46468ef7498" (UID: "f71d8c7b-94ce-4f89-9826-c46468ef7498"). InnerVolumeSpecName "kube-api-access-kp74f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:09:42.734326 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.734294 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-util" (OuterVolumeSpecName: "util") pod "f71d8c7b-94ce-4f89-9826-c46468ef7498" (UID: "f71d8c7b-94ce-4f89-9826-c46468ef7498"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:09:42.830564 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.830532 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:09:42.830564 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.830561 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp74f\" (UniqueName: \"kubernetes.io/projected/f71d8c7b-94ce-4f89-9826-c46468ef7498-kube-api-access-kp74f\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:09:42.830564 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:42.830571 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f71d8c7b-94ce-4f89-9826-c46468ef7498-util\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:09:43.480052 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:43.480020 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" event={"ID":"f71d8c7b-94ce-4f89-9826-c46468ef7498","Type":"ContainerDied","Data":"4a6733f0114c7a378e4dd9b10178997602318aa9f62c334d25203f5ce1c2d35d"} Aug 03 16:09:43.480052 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:43.480048 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp7q4c" Aug 03 16:09:43.480222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:43.480073 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a6733f0114c7a378e4dd9b10178997602318aa9f62c334d25203f5ce1c2d35d" Aug 03 16:09:45.731433 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.731403 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qjcmx"] Aug 03 16:09:45.731830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.731703 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerName="pull" Aug 03 16:09:45.731830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.731713 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerName="pull" Aug 03 16:09:45.731830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.731728 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerName="extract" Aug 03 16:09:45.731830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.731734 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerName="extract" Aug 03 16:09:45.731830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.731743 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerName="util" Aug 03 16:09:45.731830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.731748 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerName="util" Aug 03 16:09:45.731830 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.731800 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f71d8c7b-94ce-4f89-9826-c46468ef7498" containerName="extract" Aug 03 16:09:45.734682 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.734664 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qjcmx" Aug 03 16:09:45.738020 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.738004 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-gsjsr\"" Aug 03 16:09:45.744687 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.744664 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qjcmx"] Aug 03 16:09:45.856938 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.856897 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24242ef7-eb4f-45e1-9959-b6f0ca92fb52-bound-sa-token\") pod \"cert-manager-55b4d954d5-qjcmx\" (UID: \"24242ef7-eb4f-45e1-9959-b6f0ca92fb52\") " pod="cert-manager/cert-manager-55b4d954d5-qjcmx" Aug 03 16:09:45.856938 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.856943 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmmpg\" (UniqueName: \"kubernetes.io/projected/24242ef7-eb4f-45e1-9959-b6f0ca92fb52-kube-api-access-hmmpg\") pod \"cert-manager-55b4d954d5-qjcmx\" (UID: \"24242ef7-eb4f-45e1-9959-b6f0ca92fb52\") " pod="cert-manager/cert-manager-55b4d954d5-qjcmx" Aug 03 16:09:45.957923 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.957883 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24242ef7-eb4f-45e1-9959-b6f0ca92fb52-bound-sa-token\") pod \"cert-manager-55b4d954d5-qjcmx\" (UID: \"24242ef7-eb4f-45e1-9959-b6f0ca92fb52\") " pod="cert-manager/cert-manager-55b4d954d5-qjcmx" Aug 03 16:09:45.957923 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.957927 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmmpg\" (UniqueName: \"kubernetes.io/projected/24242ef7-eb4f-45e1-9959-b6f0ca92fb52-kube-api-access-hmmpg\") pod \"cert-manager-55b4d954d5-qjcmx\" (UID: \"24242ef7-eb4f-45e1-9959-b6f0ca92fb52\") " pod="cert-manager/cert-manager-55b4d954d5-qjcmx" Aug 03 16:09:45.967978 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.967956 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24242ef7-eb4f-45e1-9959-b6f0ca92fb52-bound-sa-token\") pod \"cert-manager-55b4d954d5-qjcmx\" (UID: \"24242ef7-eb4f-45e1-9959-b6f0ca92fb52\") " pod="cert-manager/cert-manager-55b4d954d5-qjcmx" Aug 03 16:09:45.968089 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:45.968060 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmmpg\" (UniqueName: \"kubernetes.io/projected/24242ef7-eb4f-45e1-9959-b6f0ca92fb52-kube-api-access-hmmpg\") pod \"cert-manager-55b4d954d5-qjcmx\" (UID: \"24242ef7-eb4f-45e1-9959-b6f0ca92fb52\") " pod="cert-manager/cert-manager-55b4d954d5-qjcmx" Aug 03 16:09:46.044106 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:46.044081 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qjcmx" Aug 03 16:09:46.165977 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:46.165956 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qjcmx"] Aug 03 16:09:46.167946 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:09:46.167924 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24242ef7_eb4f_45e1_9959_b6f0ca92fb52.slice/crio-0ef8a5530a9581650823ed362c0fa388681021037f8a9e7de8ac7eeb1b9e27de WatchSource:0}: Error finding container 0ef8a5530a9581650823ed362c0fa388681021037f8a9e7de8ac7eeb1b9e27de: Status 404 returned error can't find the container with id 0ef8a5530a9581650823ed362c0fa388681021037f8a9e7de8ac7eeb1b9e27de Aug 03 16:09:46.472093 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:46.472007 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4h6lb" Aug 03 16:09:46.493587 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:46.493552 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qjcmx" event={"ID":"24242ef7-eb4f-45e1-9959-b6f0ca92fb52","Type":"ContainerStarted","Data":"d2d09fb3cde54ed1af6b3ced1cc45bf8b6710fb9a8d4bf45b7470c0d98644fdb"} Aug 03 16:09:46.493587 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:46.493591 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qjcmx" event={"ID":"24242ef7-eb4f-45e1-9959-b6f0ca92fb52","Type":"ContainerStarted","Data":"0ef8a5530a9581650823ed362c0fa388681021037f8a9e7de8ac7eeb1b9e27de"} Aug 03 16:09:46.512156 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:46.512107 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-qjcmx" podStartSLOduration=1.512090523 podStartE2EDuration="1.512090523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:09:46.511214805 +0000 UTC m=+325.069297391" watchObservedRunningTime="2026-08-03 16:09:46.512090523 +0000 UTC m=+325.070173109" Aug 03 16:09:49.410261 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.410227 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7"] Aug 03 16:09:49.412375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.412359 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" Aug 03 16:09:49.415160 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.415141 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 03 16:09:49.416569 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.416551 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 03 16:09:49.416667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.416581 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-w4dh8\"" Aug 03 16:09:49.421409 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.421388 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7"] Aug 03 16:09:49.485970 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.485942 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6kct\" (UniqueName: \"kubernetes.io/projected/d881ab55-1857-4390-b46a-b1c10a53b5cc-kube-api-access-f6kct\") pod \"openshift-lws-operator-fd8ccff4c-sjrf7\" (UID: \"d881ab55-1857-4390-b46a-b1c10a53b5cc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" Aug 03 16:09:49.486136 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.486010 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d881ab55-1857-4390-b46a-b1c10a53b5cc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sjrf7\" (UID: \"d881ab55-1857-4390-b46a-b1c10a53b5cc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" Aug 03 16:09:49.587395 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.587360 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d881ab55-1857-4390-b46a-b1c10a53b5cc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sjrf7\" (UID: \"d881ab55-1857-4390-b46a-b1c10a53b5cc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" Aug 03 16:09:49.587531 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.587416 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6kct\" (UniqueName: \"kubernetes.io/projected/d881ab55-1857-4390-b46a-b1c10a53b5cc-kube-api-access-f6kct\") pod \"openshift-lws-operator-fd8ccff4c-sjrf7\" (UID: \"d881ab55-1857-4390-b46a-b1c10a53b5cc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" Aug 03 16:09:49.587849 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.587831 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d881ab55-1857-4390-b46a-b1c10a53b5cc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sjrf7\" (UID: \"d881ab55-1857-4390-b46a-b1c10a53b5cc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" Aug 03 16:09:49.598668 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.598649 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6kct\" (UniqueName: \"kubernetes.io/projected/d881ab55-1857-4390-b46a-b1c10a53b5cc-kube-api-access-f6kct\") pod \"openshift-lws-operator-fd8ccff4c-sjrf7\" (UID: \"d881ab55-1857-4390-b46a-b1c10a53b5cc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" Aug 03 16:09:49.722522 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.722442 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" Aug 03 16:09:49.868437 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:49.868411 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7"] Aug 03 16:09:49.871516 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:09:49.871490 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd881ab55_1857_4390_b46a_b1c10a53b5cc.slice/crio-08eb76b6a06ce75b0b93d444a298255659cfa012b722c768db07450643f8852e WatchSource:0}: Error finding container 08eb76b6a06ce75b0b93d444a298255659cfa012b722c768db07450643f8852e: Status 404 returned error can't find the container with id 08eb76b6a06ce75b0b93d444a298255659cfa012b722c768db07450643f8852e Aug 03 16:09:50.511140 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:50.511108 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" event={"ID":"d881ab55-1857-4390-b46a-b1c10a53b5cc","Type":"ContainerStarted","Data":"08eb76b6a06ce75b0b93d444a298255659cfa012b722c768db07450643f8852e"} Aug 03 16:09:53.525593 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:53.525566 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" event={"ID":"d881ab55-1857-4390-b46a-b1c10a53b5cc","Type":"ContainerStarted","Data":"492a09cc901edea191bd3220b35ac186c8a4084218dcc96e7dab95b552919143"} Aug 03 16:09:54.550905 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:54.550855 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sjrf7" podStartSLOduration=1.9922089029999999 podStartE2EDuration="5.550839042s" totalImagesPullingTime="3.558630139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:09:54.54928002 +0000 UTC m=+333.107362606" watchObservedRunningTime="2026-08-03 16:09:54.550839042 +0000 UTC m=+333.108921626" Aug 03 16:09:56.814224 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:56.814195 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h"] Aug 03 16:09:56.836513 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:56.836485 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h"] Aug 03 16:09:56.836691 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:56.836578 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:56.839497 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:56.839478 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 03 16:09:56.840944 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:56.840926 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 03 16:09:56.841056 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:56.840975 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fqmmj\"" Aug 03 16:09:56.942888 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:56.942855 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qk6j\" (UniqueName: \"kubernetes.io/projected/3a59b458-d90d-4191-9109-53cc4085ea1a-kube-api-access-8qk6j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:56.943062 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:56.942901 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:56.943062 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:56.943014 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:57.043646 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.043599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:57.043804 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.043782 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qk6j\" (UniqueName: \"kubernetes.io/projected/3a59b458-d90d-4191-9109-53cc4085ea1a-kube-api-access-8qk6j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:57.043865 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.043829 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:57.043982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.043964 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:57.044144 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.044125 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:57.052726 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.052700 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qk6j\" (UniqueName: \"kubernetes.io/projected/3a59b458-d90d-4191-9109-53cc4085ea1a-kube-api-access-8qk6j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:57.145954 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.145864 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:09:57.292304 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.292279 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h"] Aug 03 16:09:57.294023 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:09:57.294000 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a59b458_d90d_4191_9109_53cc4085ea1a.slice/crio-a6d5378297dcf8c0018056fb84d813c90895b67a63f7cce79d10a2e29fc2ca42 WatchSource:0}: Error finding container a6d5378297dcf8c0018056fb84d813c90895b67a63f7cce79d10a2e29fc2ca42: Status 404 returned error can't find the container with id a6d5378297dcf8c0018056fb84d813c90895b67a63f7cce79d10a2e29fc2ca42 Aug 03 16:09:57.543878 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.543848 2582 generic.go:356] "Generic (PLEG): container finished" podID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerID="38ade183f918ba8dac5d0eafe6d1915ac54b8829f7ad211ebefb22bb0812d632" exitCode=0 Aug 03 16:09:57.544035 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.543911 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" event={"ID":"3a59b458-d90d-4191-9109-53cc4085ea1a","Type":"ContainerDied","Data":"38ade183f918ba8dac5d0eafe6d1915ac54b8829f7ad211ebefb22bb0812d632"} Aug 03 16:09:57.544035 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:57.543955 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" event={"ID":"3a59b458-d90d-4191-9109-53cc4085ea1a","Type":"ContainerStarted","Data":"a6d5378297dcf8c0018056fb84d813c90895b67a63f7cce79d10a2e29fc2ca42"} Aug 03 16:09:58.550229 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:58.550193 2582 generic.go:356] "Generic (PLEG): container finished" podID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerID="681214cbaf1524d174654d7ac12911b762bf03ee79323f9e74eda9cf021998bd" exitCode=0 Aug 03 16:09:58.550672 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:58.550276 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" event={"ID":"3a59b458-d90d-4191-9109-53cc4085ea1a","Type":"ContainerDied","Data":"681214cbaf1524d174654d7ac12911b762bf03ee79323f9e74eda9cf021998bd"} Aug 03 16:09:59.557049 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:59.557018 2582 generic.go:356] "Generic (PLEG): container finished" podID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerID="981e6f0a0e5b87846eff8285c2e67cc961223e08978722d1902c117e88fac544" exitCode=0 Aug 03 16:09:59.557439 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:09:59.557100 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" event={"ID":"3a59b458-d90d-4191-9109-53cc4085ea1a","Type":"ContainerDied","Data":"981e6f0a0e5b87846eff8285c2e67cc961223e08978722d1902c117e88fac544"} Aug 03 16:10:00.683631 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.683592 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:10:00.773525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.773496 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qk6j\" (UniqueName: \"kubernetes.io/projected/3a59b458-d90d-4191-9109-53cc4085ea1a-kube-api-access-8qk6j\") pod \"3a59b458-d90d-4191-9109-53cc4085ea1a\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " Aug 03 16:10:00.773727 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.773553 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-util\") pod \"3a59b458-d90d-4191-9109-53cc4085ea1a\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " Aug 03 16:10:00.773727 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.773636 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-bundle\") pod \"3a59b458-d90d-4191-9109-53cc4085ea1a\" (UID: \"3a59b458-d90d-4191-9109-53cc4085ea1a\") " Aug 03 16:10:00.774353 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.774319 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-bundle" (OuterVolumeSpecName: "bundle") pod "3a59b458-d90d-4191-9109-53cc4085ea1a" (UID: "3a59b458-d90d-4191-9109-53cc4085ea1a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:10:00.775578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.775555 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a59b458-d90d-4191-9109-53cc4085ea1a-kube-api-access-8qk6j" (OuterVolumeSpecName: "kube-api-access-8qk6j") pod "3a59b458-d90d-4191-9109-53cc4085ea1a" (UID: "3a59b458-d90d-4191-9109-53cc4085ea1a"). InnerVolumeSpecName "kube-api-access-8qk6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:10:00.779037 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.779015 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-util" (OuterVolumeSpecName: "util") pod "3a59b458-d90d-4191-9109-53cc4085ea1a" (UID: "3a59b458-d90d-4191-9109-53cc4085ea1a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:10:00.875254 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.875173 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qk6j\" (UniqueName: \"kubernetes.io/projected/3a59b458-d90d-4191-9109-53cc4085ea1a-kube-api-access-8qk6j\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:10:00.875254 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.875207 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-util\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:10:00.875254 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:00.875216 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a59b458-d90d-4191-9109-53cc4085ea1a-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:10:01.568387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:01.568359 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" Aug 03 16:10:01.568387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:01.568368 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx2b9h" event={"ID":"3a59b458-d90d-4191-9109-53cc4085ea1a","Type":"ContainerDied","Data":"a6d5378297dcf8c0018056fb84d813c90895b67a63f7cce79d10a2e29fc2ca42"} Aug 03 16:10:01.568585 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:01.568412 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6d5378297dcf8c0018056fb84d813c90895b67a63f7cce79d10a2e29fc2ca42" Aug 03 16:10:11.080010 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.079973 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc"] Aug 03 16:10:11.080367 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.080290 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerName="util" Aug 03 16:10:11.080367 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.080302 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerName="util" Aug 03 16:10:11.080367 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.080318 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerName="pull" Aug 03 16:10:11.080367 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.080323 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerName="pull" Aug 03 16:10:11.080367 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.080330 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerName="extract" Aug 03 16:10:11.080367 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.080335 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerName="extract" Aug 03 16:10:11.080559 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.080387 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a59b458-d90d-4191-9109-53cc4085ea1a" containerName="extract" Aug 03 16:10:11.087495 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.087467 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.090271 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.090246 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 03 16:10:11.090398 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.090323 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6rvp9\"" Aug 03 16:10:11.090465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.090400 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 03 16:10:11.090716 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.090701 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 03 16:10:11.090798 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.090724 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 03 16:10:11.099350 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.099322 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc"] Aug 03 16:10:11.161849 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.161822 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/618e12cf-4fbb-4dfb-ad05-2095f55819fc-manifests\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.161849 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.161858 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/618e12cf-4fbb-4dfb-ad05-2095f55819fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.162076 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.161879 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxh5v\" (UniqueName: \"kubernetes.io/projected/618e12cf-4fbb-4dfb-ad05-2095f55819fc-kube-api-access-wxh5v\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.162076 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.161920 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/618e12cf-4fbb-4dfb-ad05-2095f55819fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.162076 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.161969 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/618e12cf-4fbb-4dfb-ad05-2095f55819fc-tmp\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.262933 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.262905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/618e12cf-4fbb-4dfb-ad05-2095f55819fc-manifests\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.263113 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.262942 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/618e12cf-4fbb-4dfb-ad05-2095f55819fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.263113 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.262963 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxh5v\" (UniqueName: \"kubernetes.io/projected/618e12cf-4fbb-4dfb-ad05-2095f55819fc-kube-api-access-wxh5v\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.263113 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.262997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/618e12cf-4fbb-4dfb-ad05-2095f55819fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.263113 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.263041 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/618e12cf-4fbb-4dfb-ad05-2095f55819fc-tmp\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.263386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.263360 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/618e12cf-4fbb-4dfb-ad05-2095f55819fc-manifests\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.265561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.265535 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/618e12cf-4fbb-4dfb-ad05-2095f55819fc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.265670 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.265613 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/618e12cf-4fbb-4dfb-ad05-2095f55819fc-tmp\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.265867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.265848 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/618e12cf-4fbb-4dfb-ad05-2095f55819fc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.272224 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.272198 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxh5v\" (UniqueName: \"kubernetes.io/projected/618e12cf-4fbb-4dfb-ad05-2095f55819fc-kube-api-access-wxh5v\") pod \"opendatahub-operator-controller-manager-7c4d846dc9-c2wvc\" (UID: \"618e12cf-4fbb-4dfb-ad05-2095f55819fc\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.397503 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.397417 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:11.534970 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.534944 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc"] Aug 03 16:10:11.538795 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:10:11.538766 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod618e12cf_4fbb_4dfb_ad05_2095f55819fc.slice/crio-1987eb8fe33692a3f1037a3af33bca09a86700028b5ca3de668eca5539a6192e WatchSource:0}: Error finding container 1987eb8fe33692a3f1037a3af33bca09a86700028b5ca3de668eca5539a6192e: Status 404 returned error can't find the container with id 1987eb8fe33692a3f1037a3af33bca09a86700028b5ca3de668eca5539a6192e Aug 03 16:10:11.610940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:11.610908 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" event={"ID":"618e12cf-4fbb-4dfb-ad05-2095f55819fc","Type":"ContainerStarted","Data":"1987eb8fe33692a3f1037a3af33bca09a86700028b5ca3de668eca5539a6192e"} Aug 03 16:10:15.637860 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:15.637817 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" event={"ID":"618e12cf-4fbb-4dfb-ad05-2095f55819fc","Type":"ContainerStarted","Data":"a6bb5a95625a50677cdaeaf2429b1597c4eb734cef2a1805fea62df84fdca738"} Aug 03 16:10:16.644472 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:16.644439 2582 generic.go:356] "Generic (PLEG): container finished" podID="618e12cf-4fbb-4dfb-ad05-2095f55819fc" containerID="a6bb5a95625a50677cdaeaf2429b1597c4eb734cef2a1805fea62df84fdca738" exitCode=0 Aug 03 16:10:16.644861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:16.644486 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" event={"ID":"618e12cf-4fbb-4dfb-ad05-2095f55819fc","Type":"ContainerDied","Data":"a6bb5a95625a50677cdaeaf2429b1597c4eb734cef2a1805fea62df84fdca738"} Aug 03 16:10:17.650600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:17.650561 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" event={"ID":"618e12cf-4fbb-4dfb-ad05-2095f55819fc","Type":"ContainerStarted","Data":"6cc9ab7a9d79e40d14a708da0fcdcb35fa6759a9f64e336b300238c46635bffe"} Aug 03 16:10:17.651004 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:17.650673 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:17.677094 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:17.677051 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" podStartSLOduration=1.950576093 podStartE2EDuration="6.677037043s" totalImagesPullingTime="3.696248195s" totalInitContainerRuntime="1.030212755s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:17.674024129 +0000 UTC m=+356.232106726" watchObservedRunningTime="2026-08-03 16:10:17.677037043 +0000 UTC m=+356.235119627" Aug 03 16:10:28.658257 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:28.658228 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4d846dc9-c2wvc" Aug 03 16:10:31.900396 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:31.900359 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6xmkz"] Aug 03 16:10:31.910208 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:31.910189 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:31.913183 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:31.913166 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Aug 03 16:10:31.913183 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:31.913179 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-hnwxs\"" Aug 03 16:10:31.920635 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:31.919593 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6xmkz"] Aug 03 16:10:32.044452 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:32.044416 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6335474c-f23a-45a3-a678-cd809b190022-tls-certs\") pod \"model-serving-api-54d89bd5fc-6xmkz\" (UID: \"6335474c-f23a-45a3-a678-cd809b190022\") " pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:32.044648 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:32.044473 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdw6r\" (UniqueName: \"kubernetes.io/projected/6335474c-f23a-45a3-a678-cd809b190022-kube-api-access-kdw6r\") pod \"model-serving-api-54d89bd5fc-6xmkz\" (UID: \"6335474c-f23a-45a3-a678-cd809b190022\") " pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:32.145945 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:32.145910 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6335474c-f23a-45a3-a678-cd809b190022-tls-certs\") pod \"model-serving-api-54d89bd5fc-6xmkz\" (UID: \"6335474c-f23a-45a3-a678-cd809b190022\") " pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:32.146103 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:32.146078 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdw6r\" (UniqueName: \"kubernetes.io/projected/6335474c-f23a-45a3-a678-cd809b190022-kube-api-access-kdw6r\") pod \"model-serving-api-54d89bd5fc-6xmkz\" (UID: \"6335474c-f23a-45a3-a678-cd809b190022\") " pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:32.148255 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:32.148229 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6335474c-f23a-45a3-a678-cd809b190022-tls-certs\") pod \"model-serving-api-54d89bd5fc-6xmkz\" (UID: \"6335474c-f23a-45a3-a678-cd809b190022\") " pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:32.155810 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:32.155757 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdw6r\" (UniqueName: \"kubernetes.io/projected/6335474c-f23a-45a3-a678-cd809b190022-kube-api-access-kdw6r\") pod \"model-serving-api-54d89bd5fc-6xmkz\" (UID: \"6335474c-f23a-45a3-a678-cd809b190022\") " pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:32.227811 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:32.227771 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:32.355287 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:32.355262 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6xmkz"] Aug 03 16:10:32.726581 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:32.726546 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" event={"ID":"6335474c-f23a-45a3-a678-cd809b190022","Type":"ContainerStarted","Data":"d5507f4f8cbf828dfa65a1bf1d5133a8e969d9761ef42d1cdf3bcde36257f06f"} Aug 03 16:10:35.742482 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:35.742448 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" event={"ID":"6335474c-f23a-45a3-a678-cd809b190022","Type":"ContainerStarted","Data":"9a453a292be3e6ef4bd96bd1a8a033d143ad4f4c5a86cfc9b228179e558db9ba"} Aug 03 16:10:35.742883 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:35.742506 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:35.767596 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:35.767546 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" podStartSLOduration=2.428863902 podStartE2EDuration="4.767531802s" totalImagesPullingTime="2.3386679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:35.765932298 +0000 UTC m=+374.324014882" watchObservedRunningTime="2026-08-03 16:10:35.767531802 +0000 UTC m=+374.325614429" Aug 03 16:10:37.814824 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:37.814788 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d558fbc7f-j2h6x"] Aug 03 16:10:41.028724 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.028695 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8"] Aug 03 16:10:41.032021 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.032005 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.036667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.036644 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 03 16:10:41.036782 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.036642 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 03 16:10:41.038231 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.038214 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-nhq9j\"" Aug 03 16:10:41.038339 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.038262 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 03 16:10:41.051452 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.049990 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8"] Aug 03 16:10:41.122777 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.122743 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/584f7512-abe2-42d4-9503-778797f6f612-manager-config\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.122777 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.122789 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/584f7512-abe2-42d4-9503-778797f6f612-cert\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.122991 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.122819 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqhlk\" (UniqueName: \"kubernetes.io/projected/584f7512-abe2-42d4-9503-778797f6f612-kube-api-access-kqhlk\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.122991 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.122847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/584f7512-abe2-42d4-9503-778797f6f612-metrics-cert\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.223511 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.223481 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/584f7512-abe2-42d4-9503-778797f6f612-cert\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.223679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.223523 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqhlk\" (UniqueName: \"kubernetes.io/projected/584f7512-abe2-42d4-9503-778797f6f612-kube-api-access-kqhlk\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.223679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.223546 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/584f7512-abe2-42d4-9503-778797f6f612-metrics-cert\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.223679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.223591 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/584f7512-abe2-42d4-9503-778797f6f612-manager-config\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.224261 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.224236 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/584f7512-abe2-42d4-9503-778797f6f612-manager-config\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.225933 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.225903 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/584f7512-abe2-42d4-9503-778797f6f612-metrics-cert\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.226034 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.225988 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/584f7512-abe2-42d4-9503-778797f6f612-cert\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.234392 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.234369 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqhlk\" (UniqueName: \"kubernetes.io/projected/584f7512-abe2-42d4-9503-778797f6f612-kube-api-access-kqhlk\") pod \"lws-controller-manager-7596f8d454-5n7l8\" (UID: \"584f7512-abe2-42d4-9503-778797f6f612\") " pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.342093 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.342004 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:41.500161 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.500130 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8"] Aug 03 16:10:41.503419 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:10:41.503395 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod584f7512_abe2_42d4_9503_778797f6f612.slice/crio-12cb4bbef6cfa69d756abf01e9b8e491075139d7ca024670c27f750018141339 WatchSource:0}: Error finding container 12cb4bbef6cfa69d756abf01e9b8e491075139d7ca024670c27f750018141339: Status 404 returned error can't find the container with id 12cb4bbef6cfa69d756abf01e9b8e491075139d7ca024670c27f750018141339 Aug 03 16:10:41.772491 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:41.772462 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" event={"ID":"584f7512-abe2-42d4-9503-778797f6f612","Type":"ContainerStarted","Data":"12cb4bbef6cfa69d756abf01e9b8e491075139d7ca024670c27f750018141339"} Aug 03 16:10:43.785705 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:43.785663 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" event={"ID":"584f7512-abe2-42d4-9503-778797f6f612","Type":"ContainerStarted","Data":"a2a351708f4b17bf2ffad4594aa4f83f6d59272e1828dc7b0b2a542daa84e3cd"} Aug 03 16:10:43.786479 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:43.786455 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:10:43.811688 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:43.811601 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" podStartSLOduration=1.235580735 podStartE2EDuration="2.811575455s" totalImagesPullingTime="1.57599472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:43.810976843 +0000 UTC m=+382.369059430" watchObservedRunningTime="2026-08-03 16:10:43.811575455 +0000 UTC m=+382.369658040" Aug 03 16:10:46.752375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:46.752345 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6xmkz" Aug 03 16:10:49.856041 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.856012 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg"] Aug 03 16:10:49.862353 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.862332 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.867160 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.867135 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 03 16:10:49.867288 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.867267 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 03 16:10:49.867353 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.867283 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 03 16:10:49.867353 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.867312 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kphqf\"" Aug 03 16:10:49.874954 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.874932 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg"] Aug 03 16:10:49.896940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.896907 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.897093 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.896945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.897093 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.897028 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8758d0d7-c58c-4266-b081-d8d598272127-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.897093 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.897085 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.897198 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.897114 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.897198 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.897133 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.897198 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.897183 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8758d0d7-c58c-4266-b081-d8d598272127-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.897281 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.897242 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8758d0d7-c58c-4266-b081-d8d598272127-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.897281 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.897265 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kblp\" (UniqueName: \"kubernetes.io/projected/8758d0d7-c58c-4266-b081-d8d598272127-kube-api-access-9kblp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998265 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998265 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998268 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998489 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998284 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998489 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998301 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8758d0d7-c58c-4266-b081-d8d598272127-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998489 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998331 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8758d0d7-c58c-4266-b081-d8d598272127-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998489 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kblp\" (UniqueName: \"kubernetes.io/projected/8758d0d7-c58c-4266-b081-d8d598272127-kube-api-access-9kblp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998489 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998377 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998756 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998568 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998756 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998685 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998756 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998686 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8758d0d7-c58c-4266-b081-d8d598272127-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998756 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998717 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998949 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998817 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.998949 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.998920 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:49.999169 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:49.999153 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8758d0d7-c58c-4266-b081-d8d598272127-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:50.000682 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:50.000664 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8758d0d7-c58c-4266-b081-d8d598272127-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:50.000786 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:50.000768 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8758d0d7-c58c-4266-b081-d8d598272127-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:50.008009 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:50.007982 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8758d0d7-c58c-4266-b081-d8d598272127-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:50.008112 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:50.008083 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kblp\" (UniqueName: \"kubernetes.io/projected/8758d0d7-c58c-4266-b081-d8d598272127-kube-api-access-9kblp\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg\" (UID: \"8758d0d7-c58c-4266-b081-d8d598272127\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:50.171764 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:50.171677 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:50.301746 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:50.301721 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg"] Aug 03 16:10:50.303094 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:10:50.303069 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8758d0d7_c58c_4266_b081_d8d598272127.slice/crio-11c242ed1399c149806e9fdfa71e55672f6830247a63b57912fe2eb58aad5eb1 WatchSource:0}: Error finding container 11c242ed1399c149806e9fdfa71e55672f6830247a63b57912fe2eb58aad5eb1: Status 404 returned error can't find the container with id 11c242ed1399c149806e9fdfa71e55672f6830247a63b57912fe2eb58aad5eb1 Aug 03 16:10:50.823518 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:50.823476 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" event={"ID":"8758d0d7-c58c-4266-b081-d8d598272127","Type":"ContainerStarted","Data":"11c242ed1399c149806e9fdfa71e55672f6830247a63b57912fe2eb58aad5eb1"} Aug 03 16:10:53.137090 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:53.137051 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:10:53.137430 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:53.137137 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:10:53.137430 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:53.137181 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:10:53.845150 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:53.845109 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" event={"ID":"8758d0d7-c58c-4266-b081-d8d598272127","Type":"ContainerStarted","Data":"d0f8df68f64938e73a70fb1b59840a05d8bff61abdf08250ab7d9fe5a5dd3b4f"} Aug 03 16:10:53.865633 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:53.865566 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" podStartSLOduration=2.033666729 podStartE2EDuration="4.865549851s" totalImagesPullingTime="2.831883122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:10:53.864964019 +0000 UTC m=+392.423046603" watchObservedRunningTime="2026-08-03 16:10:53.865549851 +0000 UTC m=+392.423632437" Aug 03 16:10:54.171976 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:54.171896 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:54.305091 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:54.305061 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:54.850367 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:54.850338 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:54.851226 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:54.851203 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg" Aug 03 16:10:55.796642 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:10:55.796584 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7596f8d454-5n7l8" Aug 03 16:11:02.834892 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:02.834835 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d558fbc7f-j2h6x" podUID="78c5a66a-ec47-4dfe-9a50-7c3190662a1d" containerName="console" containerID="cri-o://a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a" gracePeriod=15 Aug 03 16:11:03.079800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.079776 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:11:03.214871 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.214789 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-oauth-config\") pod \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " Aug 03 16:11:03.214871 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.214825 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-config\") pod \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " Aug 03 16:11:03.214871 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.214847 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8g78\" (UniqueName: \"kubernetes.io/projected/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-kube-api-access-f8g78\") pod \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " Aug 03 16:11:03.214871 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.214861 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-serving-cert\") pod \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " Aug 03 16:11:03.215111 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.215069 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-trusted-ca-bundle\") pod \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " Aug 03 16:11:03.215162 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.215113 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-oauth-serving-cert\") pod \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " Aug 03 16:11:03.215222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.215208 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-service-ca\") pod \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\" (UID: \"78c5a66a-ec47-4dfe-9a50-7c3190662a1d\") " Aug 03 16:11:03.215332 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.215264 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-config" (OuterVolumeSpecName: "console-config") pod "78c5a66a-ec47-4dfe-9a50-7c3190662a1d" (UID: "78c5a66a-ec47-4dfe-9a50-7c3190662a1d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:11:03.215492 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.215472 2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:03.215555 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.215533 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "78c5a66a-ec47-4dfe-9a50-7c3190662a1d" (UID: "78c5a66a-ec47-4dfe-9a50-7c3190662a1d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:11:03.215555 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.215538 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "78c5a66a-ec47-4dfe-9a50-7c3190662a1d" (UID: "78c5a66a-ec47-4dfe-9a50-7c3190662a1d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:11:03.215656 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.215588 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-service-ca" (OuterVolumeSpecName: "service-ca") pod "78c5a66a-ec47-4dfe-9a50-7c3190662a1d" (UID: "78c5a66a-ec47-4dfe-9a50-7c3190662a1d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:11:03.217106 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.217086 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "78c5a66a-ec47-4dfe-9a50-7c3190662a1d" (UID: "78c5a66a-ec47-4dfe-9a50-7c3190662a1d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:11:03.217406 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.217386 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-kube-api-access-f8g78" (OuterVolumeSpecName: "kube-api-access-f8g78") pod "78c5a66a-ec47-4dfe-9a50-7c3190662a1d" (UID: "78c5a66a-ec47-4dfe-9a50-7c3190662a1d"). InnerVolumeSpecName "kube-api-access-f8g78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:11:03.217406 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.217395 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "78c5a66a-ec47-4dfe-9a50-7c3190662a1d" (UID: "78c5a66a-ec47-4dfe-9a50-7c3190662a1d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:11:03.316743 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.316712 2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-oauth-config\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:03.316743 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.316736 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8g78\" (UniqueName: \"kubernetes.io/projected/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-kube-api-access-f8g78\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:03.316743 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.316747 2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-console-serving-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:03.316971 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.316756 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-trusted-ca-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:03.316971 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.316764 2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-oauth-serving-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:03.316971 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.316772 2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/78c5a66a-ec47-4dfe-9a50-7c3190662a1d-service-ca\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:03.894282 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.894250 2582 generic.go:356] "Generic (PLEG): container finished" podID="78c5a66a-ec47-4dfe-9a50-7c3190662a1d" containerID="a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a" exitCode=0 Aug 03 16:11:03.894757 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.894295 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d558fbc7f-j2h6x" Aug 03 16:11:03.894757 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.894314 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d558fbc7f-j2h6x" event={"ID":"78c5a66a-ec47-4dfe-9a50-7c3190662a1d","Type":"ContainerDied","Data":"a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a"} Aug 03 16:11:03.894757 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.894374 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d558fbc7f-j2h6x" event={"ID":"78c5a66a-ec47-4dfe-9a50-7c3190662a1d","Type":"ContainerDied","Data":"d9d9dc227ef3b622be32193e2a717ea8fe18aaa2139e08b9cabf3df7afbabecd"} Aug 03 16:11:03.894757 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.894395 2582 scope.go:117] "RemoveContainer" containerID="a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a" Aug 03 16:11:03.903079 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.903062 2582 scope.go:117] "RemoveContainer" containerID="a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a" Aug 03 16:11:03.903338 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:11:03.903321 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a\": container with ID starting with a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a not found: ID does not exist" containerID="a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a" Aug 03 16:11:03.903399 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.903350 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a"} err="failed to get container status \"a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a\": rpc error: code = NotFound desc = could not find container \"a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a\": container with ID starting with a3a56ba447d6c3a7bc88e9a105b162d5f611bf200ce609bf106934c06c52a19a not found: ID does not exist" Aug 03 16:11:03.917362 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.917331 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d558fbc7f-j2h6x"] Aug 03 16:11:03.925061 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.925040 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d558fbc7f-j2h6x"] Aug 03 16:11:03.995198 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:03.995165 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78c5a66a-ec47-4dfe-9a50-7c3190662a1d" path="/var/lib/kubelet/pods/78c5a66a-ec47-4dfe-9a50-7c3190662a1d/volumes" Aug 03 16:11:35.493703 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.493671 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9"] Aug 03 16:11:35.494240 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.494062 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78c5a66a-ec47-4dfe-9a50-7c3190662a1d" containerName="console" Aug 03 16:11:35.494240 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.494076 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="78c5a66a-ec47-4dfe-9a50-7c3190662a1d" containerName="console" Aug 03 16:11:35.494240 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.494161 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="78c5a66a-ec47-4dfe-9a50-7c3190662a1d" containerName="console" Aug 03 16:11:35.502707 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.502683 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.506472 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.506303 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fqmmj\"" Aug 03 16:11:35.506472 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.506314 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 03 16:11:35.506672 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.506524 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 03 16:11:35.507535 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.507511 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9"] Aug 03 16:11:35.584312 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.584284 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.584504 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.584327 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4c6x\" (UniqueName: \"kubernetes.io/projected/7a320cf3-b9b2-49ae-818b-b86b733b0e28-kube-api-access-b4c6x\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.584504 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.584371 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.608611 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.608578 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546"] Aug 03 16:11:35.611987 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.611972 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.620982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.620962 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546"] Aug 03 16:11:35.685547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.685519 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.685751 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.685572 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p2z6\" (UniqueName: \"kubernetes.io/projected/5c291bff-6353-4030-bcb3-35518e40007d-kube-api-access-9p2z6\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.685751 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.685599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.685751 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.685659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4c6x\" (UniqueName: \"kubernetes.io/projected/7a320cf3-b9b2-49ae-818b-b86b733b0e28-kube-api-access-b4c6x\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.685751 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.685703 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.685963 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.685821 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.685963 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.685921 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.686056 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.685977 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.691651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.691606 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq"] Aug 03 16:11:35.695450 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.695435 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.699336 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.699315 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4c6x\" (UniqueName: \"kubernetes.io/projected/7a320cf3-b9b2-49ae-818b-b86b733b0e28-kube-api-access-b4c6x\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.704669 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.704648 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq"] Aug 03 16:11:35.786918 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.786886 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p2z6\" (UniqueName: \"kubernetes.io/projected/5c291bff-6353-4030-bcb3-35518e40007d-kube-api-access-9p2z6\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.787104 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.786953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.787104 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.786991 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.787104 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.787015 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.787104 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.787079 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.787267 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.787102 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72m6b\" (UniqueName: \"kubernetes.io/projected/28c4be15-794e-4c15-8698-abb148fb4994-kube-api-access-72m6b\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.787429 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.787405 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.787466 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.787415 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.797641 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.797605 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p2z6\" (UniqueName: \"kubernetes.io/projected/5c291bff-6353-4030-bcb3-35518e40007d-kube-api-access-9p2z6\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.805404 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.805382 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl"] Aug 03 16:11:35.808913 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.808899 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:35.812637 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.812595 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:35.818997 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.818976 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl"] Aug 03 16:11:35.888020 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.887990 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.888020 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.888022 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72m6b\" (UniqueName: \"kubernetes.io/projected/28c4be15-794e-4c15-8698-abb148fb4994-kube-api-access-72m6b\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.888241 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.888072 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.888241 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.888110 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqw24\" (UniqueName: \"kubernetes.io/projected/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-kube-api-access-cqw24\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:35.888241 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.888130 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:35.888241 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.888144 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:35.888441 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.888381 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.888441 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.888406 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.899198 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.899171 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72m6b\" (UniqueName: \"kubernetes.io/projected/28c4be15-794e-4c15-8698-abb148fb4994-kube-api-access-72m6b\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:35.921851 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.921826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:35.945870 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.945848 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9"] Aug 03 16:11:35.947930 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:11:35.947906 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a320cf3_b9b2_49ae_818b_b86b733b0e28.slice/crio-befdcc85467381b868bd7c9031fb9e46d99b2c826bd56992b963ea9cb45c81c9 WatchSource:0}: Error finding container befdcc85467381b868bd7c9031fb9e46d99b2c826bd56992b963ea9cb45c81c9: Status 404 returned error can't find the container with id befdcc85467381b868bd7c9031fb9e46d99b2c826bd56992b963ea9cb45c81c9 Aug 03 16:11:35.989643 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.989585 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqw24\" (UniqueName: \"kubernetes.io/projected/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-kube-api-access-cqw24\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:35.989643 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.989637 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:35.989825 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.989655 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:35.989995 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.989979 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:35.990043 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:35.990014 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:36.000068 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:36.000045 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqw24\" (UniqueName: \"kubernetes.io/projected/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-kube-api-access-cqw24\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:36.011963 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:36.011938 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:36.045153 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:36.045058 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" event={"ID":"7a320cf3-b9b2-49ae-818b-b86b733b0e28","Type":"ContainerStarted","Data":"cca80ca6a492e7f89f2d265224cf2189225866dec5ef25fdfe56d10408cd94c5"} Aug 03 16:11:36.045153 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:36.045115 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" event={"ID":"7a320cf3-b9b2-49ae-818b-b86b733b0e28","Type":"ContainerStarted","Data":"befdcc85467381b868bd7c9031fb9e46d99b2c826bd56992b963ea9cb45c81c9"} Aug 03 16:11:36.060364 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:36.060342 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546"] Aug 03 16:11:36.062766 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:11:36.062726 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c291bff_6353_4030_bcb3_35518e40007d.slice/crio-058898de9d72af7cc00eddf57b500d08b85d0ca22ca422aed5cf61e6b89a5b8f WatchSource:0}: Error finding container 058898de9d72af7cc00eddf57b500d08b85d0ca22ca422aed5cf61e6b89a5b8f: Status 404 returned error can't find the container with id 058898de9d72af7cc00eddf57b500d08b85d0ca22ca422aed5cf61e6b89a5b8f Aug 03 16:11:36.118919 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:36.118894 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:36.159095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:36.159067 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq"] Aug 03 16:11:36.160971 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:11:36.160941 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c4be15_794e_4c15_8698_abb148fb4994.slice/crio-dd200c37539dd52f1f2a13a33cff4125f995d497ca9057c78211fe44268aca30 WatchSource:0}: Error finding container dd200c37539dd52f1f2a13a33cff4125f995d497ca9057c78211fe44268aca30: Status 404 returned error can't find the container with id dd200c37539dd52f1f2a13a33cff4125f995d497ca9057c78211fe44268aca30 Aug 03 16:11:36.265300 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:36.265273 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl"] Aug 03 16:11:36.293387 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:11:36.293356 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21cc2cbe_7abc_4b02_a4e7_efd250d1cd21.slice/crio-62b9d57f5b1141d9a2146908f5c4b15fb1c7b60dcb863e9c5514c2a9839d93bc WatchSource:0}: Error finding container 62b9d57f5b1141d9a2146908f5c4b15fb1c7b60dcb863e9c5514c2a9839d93bc: Status 404 returned error can't find the container with id 62b9d57f5b1141d9a2146908f5c4b15fb1c7b60dcb863e9c5514c2a9839d93bc Aug 03 16:11:37.050657 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.050607 2582 generic.go:356] "Generic (PLEG): container finished" podID="28c4be15-794e-4c15-8698-abb148fb4994" containerID="c9510b1dc0cb3d566469a867fe08db06267848427932d78eb71ce326b16578b6" exitCode=0 Aug 03 16:11:37.051103 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.050715 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" event={"ID":"28c4be15-794e-4c15-8698-abb148fb4994","Type":"ContainerDied","Data":"c9510b1dc0cb3d566469a867fe08db06267848427932d78eb71ce326b16578b6"} Aug 03 16:11:37.051103 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.050800 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" event={"ID":"28c4be15-794e-4c15-8698-abb148fb4994","Type":"ContainerStarted","Data":"dd200c37539dd52f1f2a13a33cff4125f995d497ca9057c78211fe44268aca30"} Aug 03 16:11:37.052288 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.052269 2582 generic.go:356] "Generic (PLEG): container finished" podID="5c291bff-6353-4030-bcb3-35518e40007d" containerID="eea5969e1dcb501f788c9754f763319318415ca301f6f31d5d5480287934f41a" exitCode=0 Aug 03 16:11:37.052385 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.052345 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" event={"ID":"5c291bff-6353-4030-bcb3-35518e40007d","Type":"ContainerDied","Data":"eea5969e1dcb501f788c9754f763319318415ca301f6f31d5d5480287934f41a"} Aug 03 16:11:37.052385 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.052379 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" event={"ID":"5c291bff-6353-4030-bcb3-35518e40007d","Type":"ContainerStarted","Data":"058898de9d72af7cc00eddf57b500d08b85d0ca22ca422aed5cf61e6b89a5b8f"} Aug 03 16:11:37.053817 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.053795 2582 generic.go:356] "Generic (PLEG): container finished" podID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerID="cca80ca6a492e7f89f2d265224cf2189225866dec5ef25fdfe56d10408cd94c5" exitCode=0 Aug 03 16:11:37.053869 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.053854 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" event={"ID":"7a320cf3-b9b2-49ae-818b-b86b733b0e28","Type":"ContainerDied","Data":"cca80ca6a492e7f89f2d265224cf2189225866dec5ef25fdfe56d10408cd94c5"} Aug 03 16:11:37.055353 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.055333 2582 generic.go:356] "Generic (PLEG): container finished" podID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerID="393d2029f089db22942eac0b0d85d4e75375b19c707b5696fbb6aede02082352" exitCode=0 Aug 03 16:11:37.055439 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.055399 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" event={"ID":"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21","Type":"ContainerDied","Data":"393d2029f089db22942eac0b0d85d4e75375b19c707b5696fbb6aede02082352"} Aug 03 16:11:37.055439 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:37.055422 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" event={"ID":"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21","Type":"ContainerStarted","Data":"62b9d57f5b1141d9a2146908f5c4b15fb1c7b60dcb863e9c5514c2a9839d93bc"} Aug 03 16:11:38.062003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:38.061978 2582 generic.go:356] "Generic (PLEG): container finished" podID="28c4be15-794e-4c15-8698-abb148fb4994" containerID="af7654e3305295d0caa430031490ef8bc540b388b0ea51cbfd411a0fe6155185" exitCode=0 Aug 03 16:11:38.062308 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:38.062067 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" event={"ID":"28c4be15-794e-4c15-8698-abb148fb4994","Type":"ContainerDied","Data":"af7654e3305295d0caa430031490ef8bc540b388b0ea51cbfd411a0fe6155185"} Aug 03 16:11:39.069482 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:39.069403 2582 generic.go:356] "Generic (PLEG): container finished" podID="5c291bff-6353-4030-bcb3-35518e40007d" containerID="07bc3aeaa79bc8e8a196c7ff10534e17a1235f88d89d002e79476b3b85939f84" exitCode=0 Aug 03 16:11:39.069867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:39.069479 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" event={"ID":"5c291bff-6353-4030-bcb3-35518e40007d","Type":"ContainerDied","Data":"07bc3aeaa79bc8e8a196c7ff10534e17a1235f88d89d002e79476b3b85939f84"} Aug 03 16:11:39.071395 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:39.071371 2582 generic.go:356] "Generic (PLEG): container finished" podID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerID="2b445b91ba5dfea684a086695d2ab3b9caae1a1f03e79e089edf5777b29edf8c" exitCode=0 Aug 03 16:11:39.071508 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:39.071442 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" event={"ID":"7a320cf3-b9b2-49ae-818b-b86b733b0e28","Type":"ContainerDied","Data":"2b445b91ba5dfea684a086695d2ab3b9caae1a1f03e79e089edf5777b29edf8c"} Aug 03 16:11:39.073253 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:39.073232 2582 generic.go:356] "Generic (PLEG): container finished" podID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerID="da76558fe6eb4a3e17d10f70bc95aed66103acd908bc318daa66c8e5ae8f4db6" exitCode=0 Aug 03 16:11:39.073335 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:39.073307 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" event={"ID":"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21","Type":"ContainerDied","Data":"da76558fe6eb4a3e17d10f70bc95aed66103acd908bc318daa66c8e5ae8f4db6"} Aug 03 16:11:39.075515 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:39.075498 2582 generic.go:356] "Generic (PLEG): container finished" podID="28c4be15-794e-4c15-8698-abb148fb4994" containerID="ae6322c266d0d510b71ce91f17d8b1e889e6c6706975fba7e890852044a0ea3e" exitCode=0 Aug 03 16:11:39.075680 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:39.075538 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" event={"ID":"28c4be15-794e-4c15-8698-abb148fb4994","Type":"ContainerDied","Data":"ae6322c266d0d510b71ce91f17d8b1e889e6c6706975fba7e890852044a0ea3e"} Aug 03 16:11:40.081537 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.081500 2582 generic.go:356] "Generic (PLEG): container finished" podID="5c291bff-6353-4030-bcb3-35518e40007d" containerID="2dcfdb820bbada7f76050ea9e5a16687d685814768a3e979fbc971f0a86d2197" exitCode=0 Aug 03 16:11:40.081991 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.081576 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" event={"ID":"5c291bff-6353-4030-bcb3-35518e40007d","Type":"ContainerDied","Data":"2dcfdb820bbada7f76050ea9e5a16687d685814768a3e979fbc971f0a86d2197"} Aug 03 16:11:40.083660 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.083637 2582 generic.go:356] "Generic (PLEG): container finished" podID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerID="fd3f8db32c388dbaa1d08a7739dc6907bd3982fe09682775adfff96dfad13304" exitCode=0 Aug 03 16:11:40.083772 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.083708 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" event={"ID":"7a320cf3-b9b2-49ae-818b-b86b733b0e28","Type":"ContainerDied","Data":"fd3f8db32c388dbaa1d08a7739dc6907bd3982fe09682775adfff96dfad13304"} Aug 03 16:11:40.085483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.085464 2582 generic.go:356] "Generic (PLEG): container finished" podID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerID="6943c5cb2d4a66f67afd12730ea209b1eccd743ffda1c98c03968a64ff40cc7c" exitCode=0 Aug 03 16:11:40.085574 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.085557 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" event={"ID":"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21","Type":"ContainerDied","Data":"6943c5cb2d4a66f67afd12730ea209b1eccd743ffda1c98c03968a64ff40cc7c"} Aug 03 16:11:40.216092 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.216070 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:40.326364 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.326332 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-util\") pod \"28c4be15-794e-4c15-8698-abb148fb4994\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " Aug 03 16:11:40.326531 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.326381 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-bundle\") pod \"28c4be15-794e-4c15-8698-abb148fb4994\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " Aug 03 16:11:40.326531 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.326432 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72m6b\" (UniqueName: \"kubernetes.io/projected/28c4be15-794e-4c15-8698-abb148fb4994-kube-api-access-72m6b\") pod \"28c4be15-794e-4c15-8698-abb148fb4994\" (UID: \"28c4be15-794e-4c15-8698-abb148fb4994\") " Aug 03 16:11:40.327041 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.327010 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-bundle" (OuterVolumeSpecName: "bundle") pod "28c4be15-794e-4c15-8698-abb148fb4994" (UID: "28c4be15-794e-4c15-8698-abb148fb4994"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:11:40.328518 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.328493 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c4be15-794e-4c15-8698-abb148fb4994-kube-api-access-72m6b" (OuterVolumeSpecName: "kube-api-access-72m6b") pod "28c4be15-794e-4c15-8698-abb148fb4994" (UID: "28c4be15-794e-4c15-8698-abb148fb4994"). InnerVolumeSpecName "kube-api-access-72m6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:11:40.330971 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.330952 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-util" (OuterVolumeSpecName: "util") pod "28c4be15-794e-4c15-8698-abb148fb4994" (UID: "28c4be15-794e-4c15-8698-abb148fb4994"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:11:40.427791 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.427759 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-util\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:40.427791 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.427784 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28c4be15-794e-4c15-8698-abb148fb4994-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:40.427791 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:40.427792 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72m6b\" (UniqueName: \"kubernetes.io/projected/28c4be15-794e-4c15-8698-abb148fb4994-kube-api-access-72m6b\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:41.092586 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.092557 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" Aug 03 16:11:41.093008 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.092600 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053jvxkq" event={"ID":"28c4be15-794e-4c15-8698-abb148fb4994","Type":"ContainerDied","Data":"dd200c37539dd52f1f2a13a33cff4125f995d497ca9057c78211fe44268aca30"} Aug 03 16:11:41.093008 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.092662 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd200c37539dd52f1f2a13a33cff4125f995d497ca9057c78211fe44268aca30" Aug 03 16:11:41.235420 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.235397 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:41.279468 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.279448 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:41.282597 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.282583 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:41.336251 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336224 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-bundle\") pod \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " Aug 03 16:11:41.336251 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336255 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-bundle\") pod \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " Aug 03 16:11:41.336487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336272 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-util\") pod \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " Aug 03 16:11:41.336487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336315 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9p2z6\" (UniqueName: \"kubernetes.io/projected/5c291bff-6353-4030-bcb3-35518e40007d-kube-api-access-9p2z6\") pod \"5c291bff-6353-4030-bcb3-35518e40007d\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " Aug 03 16:11:41.336487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336331 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-util\") pod \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " Aug 03 16:11:41.336487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336361 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4c6x\" (UniqueName: \"kubernetes.io/projected/7a320cf3-b9b2-49ae-818b-b86b733b0e28-kube-api-access-b4c6x\") pod \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\" (UID: \"7a320cf3-b9b2-49ae-818b-b86b733b0e28\") " Aug 03 16:11:41.336487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336386 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqw24\" (UniqueName: \"kubernetes.io/projected/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-kube-api-access-cqw24\") pod \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\" (UID: \"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21\") " Aug 03 16:11:41.336487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336430 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-util\") pod \"5c291bff-6353-4030-bcb3-35518e40007d\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " Aug 03 16:11:41.336487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336467 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-bundle\") pod \"5c291bff-6353-4030-bcb3-35518e40007d\" (UID: \"5c291bff-6353-4030-bcb3-35518e40007d\") " Aug 03 16:11:41.336849 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336791 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-bundle" (OuterVolumeSpecName: "bundle") pod "21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" (UID: "21cc2cbe-7abc-4b02-a4e7-efd250d1cd21"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:11:41.336905 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.336856 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-bundle" (OuterVolumeSpecName: "bundle") pod "7a320cf3-b9b2-49ae-818b-b86b733b0e28" (UID: "7a320cf3-b9b2-49ae-818b-b86b733b0e28"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:11:41.337508 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.337465 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-bundle" (OuterVolumeSpecName: "bundle") pod "5c291bff-6353-4030-bcb3-35518e40007d" (UID: "5c291bff-6353-4030-bcb3-35518e40007d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:11:41.338888 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.338855 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-kube-api-access-cqw24" (OuterVolumeSpecName: "kube-api-access-cqw24") pod "21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" (UID: "21cc2cbe-7abc-4b02-a4e7-efd250d1cd21"). InnerVolumeSpecName "kube-api-access-cqw24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:11:41.338986 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.338936 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c291bff-6353-4030-bcb3-35518e40007d-kube-api-access-9p2z6" (OuterVolumeSpecName: "kube-api-access-9p2z6") pod "5c291bff-6353-4030-bcb3-35518e40007d" (UID: "5c291bff-6353-4030-bcb3-35518e40007d"). InnerVolumeSpecName "kube-api-access-9p2z6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:11:41.339029 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.338977 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a320cf3-b9b2-49ae-818b-b86b733b0e28-kube-api-access-b4c6x" (OuterVolumeSpecName: "kube-api-access-b4c6x") pod "7a320cf3-b9b2-49ae-818b-b86b733b0e28" (UID: "7a320cf3-b9b2-49ae-818b-b86b733b0e28"). InnerVolumeSpecName "kube-api-access-b4c6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:11:41.341867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.341830 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-util" (OuterVolumeSpecName: "util") pod "7a320cf3-b9b2-49ae-818b-b86b733b0e28" (UID: "7a320cf3-b9b2-49ae-818b-b86b733b0e28"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:11:41.345060 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.345004 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-util" (OuterVolumeSpecName: "util") pod "21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" (UID: "21cc2cbe-7abc-4b02-a4e7-efd250d1cd21"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:11:41.345522 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.345507 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-util" (OuterVolumeSpecName: "util") pod "5c291bff-6353-4030-bcb3-35518e40007d" (UID: "5c291bff-6353-4030-bcb3-35518e40007d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:11:41.437283 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.437241 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p2z6\" (UniqueName: \"kubernetes.io/projected/5c291bff-6353-4030-bcb3-35518e40007d-kube-api-access-9p2z6\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:41.437283 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.437277 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-util\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:41.437283 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.437292 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4c6x\" (UniqueName: \"kubernetes.io/projected/7a320cf3-b9b2-49ae-818b-b86b733b0e28-kube-api-access-b4c6x\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:41.437525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.437304 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqw24\" (UniqueName: \"kubernetes.io/projected/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-kube-api-access-cqw24\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:41.437525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.437315 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-util\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:41.437525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.437324 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c291bff-6353-4030-bcb3-35518e40007d-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:41.437525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.437335 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21cc2cbe-7abc-4b02-a4e7-efd250d1cd21-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:41.437525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.437346 2582 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-bundle\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:41.437525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:41.437355 2582 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7a320cf3-b9b2-49ae-818b-b86b733b0e28-util\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:11:42.099391 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:42.099365 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" Aug 03 16:11:42.099391 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:42.099360 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cbb546" event={"ID":"5c291bff-6353-4030-bcb3-35518e40007d","Type":"ContainerDied","Data":"058898de9d72af7cc00eddf57b500d08b85d0ca22ca422aed5cf61e6b89a5b8f"} Aug 03 16:11:42.099882 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:42.099416 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="058898de9d72af7cc00eddf57b500d08b85d0ca22ca422aed5cf61e6b89a5b8f" Aug 03 16:11:42.101424 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:42.101405 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" Aug 03 16:11:42.101653 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:42.101399 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb2bnr9" event={"ID":"7a320cf3-b9b2-49ae-818b-b86b733b0e28","Type":"ContainerDied","Data":"befdcc85467381b868bd7c9031fb9e46d99b2c826bd56992b963ea9cb45c81c9"} Aug 03 16:11:42.101739 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:42.101680 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="befdcc85467381b868bd7c9031fb9e46d99b2c826bd56992b963ea9cb45c81c9" Aug 03 16:11:42.103458 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:42.103429 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" Aug 03 16:11:42.103554 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:42.103477 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0sf2gl" event={"ID":"21cc2cbe-7abc-4b02-a4e7-efd250d1cd21","Type":"ContainerDied","Data":"62b9d57f5b1141d9a2146908f5c4b15fb1c7b60dcb863e9c5514c2a9839d93bc"} Aug 03 16:11:42.103554 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:42.103517 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62b9d57f5b1141d9a2146908f5c4b15fb1c7b60dcb863e9c5514c2a9839d93bc" Aug 03 16:11:48.603289 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603213 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj"] Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603524 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerName="extract" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603534 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerName="extract" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603545 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c4be15-794e-4c15-8698-abb148fb4994" containerName="pull" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603550 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c4be15-794e-4c15-8698-abb148fb4994" containerName="pull" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603556 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c291bff-6353-4030-bcb3-35518e40007d" containerName="pull" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603562 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c291bff-6353-4030-bcb3-35518e40007d" containerName="pull" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603570 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c291bff-6353-4030-bcb3-35518e40007d" containerName="extract" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603575 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c291bff-6353-4030-bcb3-35518e40007d" containerName="extract" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603582 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerName="extract" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603586 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerName="extract" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603598 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c291bff-6353-4030-bcb3-35518e40007d" containerName="util" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603603 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c291bff-6353-4030-bcb3-35518e40007d" containerName="util" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603609 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerName="util" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603635 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerName="util" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603645 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c4be15-794e-4c15-8698-abb148fb4994" containerName="util" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603652 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c4be15-794e-4c15-8698-abb148fb4994" containerName="util" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603658 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerName="pull" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603663 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerName="pull" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603670 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c4be15-794e-4c15-8698-abb148fb4994" containerName="extract" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603674 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c4be15-794e-4c15-8698-abb148fb4994" containerName="extract" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603680 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerName="pull" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603685 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerName="pull" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603694 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerName="util" Aug 03 16:11:48.603709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603699 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerName="util" Aug 03 16:11:48.604346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603750 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c291bff-6353-4030-bcb3-35518e40007d" containerName="extract" Aug 03 16:11:48.604346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603758 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c4be15-794e-4c15-8698-abb148fb4994" containerName="extract" Aug 03 16:11:48.604346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603763 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="21cc2cbe-7abc-4b02-a4e7-efd250d1cd21" containerName="extract" Aug 03 16:11:48.604346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.603769 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a320cf3-b9b2-49ae-818b-b86b733b0e28" containerName="extract" Aug 03 16:11:48.606737 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.606719 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:11:48.609479 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.609454 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 03 16:11:48.609564 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.609462 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 03 16:11:48.609636 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.609577 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lwnjp\"" Aug 03 16:11:48.618850 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.618830 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj"] Aug 03 16:11:48.697863 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.697834 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ppj7\" (UniqueName: \"kubernetes.io/projected/26bd43fb-5229-410c-adb6-929f41fd554f-kube-api-access-7ppj7\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpprj\" (UID: \"26bd43fb-5229-410c-adb6-929f41fd554f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:11:48.698033 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.697908 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bd43fb-5229-410c-adb6-929f41fd554f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpprj\" (UID: \"26bd43fb-5229-410c-adb6-929f41fd554f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:11:48.798452 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.798414 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bd43fb-5229-410c-adb6-929f41fd554f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpprj\" (UID: \"26bd43fb-5229-410c-adb6-929f41fd554f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:11:48.798638 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.798483 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ppj7\" (UniqueName: \"kubernetes.io/projected/26bd43fb-5229-410c-adb6-929f41fd554f-kube-api-access-7ppj7\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpprj\" (UID: \"26bd43fb-5229-410c-adb6-929f41fd554f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:11:48.798809 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.798788 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bd43fb-5229-410c-adb6-929f41fd554f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpprj\" (UID: \"26bd43fb-5229-410c-adb6-929f41fd554f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:11:48.808988 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.808961 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ppj7\" (UniqueName: \"kubernetes.io/projected/26bd43fb-5229-410c-adb6-929f41fd554f-kube-api-access-7ppj7\") pod \"kuadrant-operator-controller-manager-58fc56757c-qpprj\" (UID: \"26bd43fb-5229-410c-adb6-929f41fd554f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:11:48.916356 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:48.916271 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:11:49.065408 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:49.065381 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj"] Aug 03 16:11:49.066898 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:11:49.066872 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26bd43fb_5229_410c_adb6_929f41fd554f.slice/crio-29ff64fecd42675142747ed19464cb90b6ae1bae2ea3b70cb6fd92d0746cae3d WatchSource:0}: Error finding container 29ff64fecd42675142747ed19464cb90b6ae1bae2ea3b70cb6fd92d0746cae3d: Status 404 returned error can't find the container with id 29ff64fecd42675142747ed19464cb90b6ae1bae2ea3b70cb6fd92d0746cae3d Aug 03 16:11:49.139364 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:49.139327 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" event={"ID":"26bd43fb-5229-410c-adb6-929f41fd554f","Type":"ContainerStarted","Data":"29ff64fecd42675142747ed19464cb90b6ae1bae2ea3b70cb6fd92d0746cae3d"} Aug 03 16:11:50.536476 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:50.536397 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-r8g86"] Aug 03 16:11:50.540506 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:50.540478 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" Aug 03 16:11:50.544110 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:50.544086 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Aug 03 16:11:50.544110 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:50.544099 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-8fhtt\"" Aug 03 16:11:50.551594 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:50.550739 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-r8g86"] Aug 03 16:11:50.613715 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:50.613678 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8k5p\" (UniqueName: \"kubernetes.io/projected/55013419-ac83-48b8-a590-93275df3b798-kube-api-access-p8k5p\") pod \"dns-operator-controller-manager-5985564776-r8g86\" (UID: \"55013419-ac83-48b8-a590-93275df3b798\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" Aug 03 16:11:50.715067 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:50.714968 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8k5p\" (UniqueName: \"kubernetes.io/projected/55013419-ac83-48b8-a590-93275df3b798-kube-api-access-p8k5p\") pod \"dns-operator-controller-manager-5985564776-r8g86\" (UID: \"55013419-ac83-48b8-a590-93275df3b798\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" Aug 03 16:11:50.728009 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:50.727977 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8k5p\" (UniqueName: \"kubernetes.io/projected/55013419-ac83-48b8-a590-93275df3b798-kube-api-access-p8k5p\") pod \"dns-operator-controller-manager-5985564776-r8g86\" (UID: \"55013419-ac83-48b8-a590-93275df3b798\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" Aug 03 16:11:50.855377 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:50.855042 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" Aug 03 16:11:51.508727 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:51.508702 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-r8g86"] Aug 03 16:11:51.510379 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:11:51.510349 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55013419_ac83_48b8_a590_93275df3b798.slice/crio-fe4aecccfd656ae5455691ca8cdd57142d917aeab175fccbc1ff401a4b89c38b WatchSource:0}: Error finding container fe4aecccfd656ae5455691ca8cdd57142d917aeab175fccbc1ff401a4b89c38b: Status 404 returned error can't find the container with id fe4aecccfd656ae5455691ca8cdd57142d917aeab175fccbc1ff401a4b89c38b Aug 03 16:11:52.159269 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:52.159233 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" event={"ID":"55013419-ac83-48b8-a590-93275df3b798","Type":"ContainerStarted","Data":"fe4aecccfd656ae5455691ca8cdd57142d917aeab175fccbc1ff401a4b89c38b"} Aug 03 16:11:53.165987 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:53.165959 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" event={"ID":"26bd43fb-5229-410c-adb6-929f41fd554f","Type":"ContainerStarted","Data":"cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1"} Aug 03 16:11:53.166331 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:53.166027 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:11:53.196730 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:53.196681 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" podStartSLOduration=1.2038238350000001 podStartE2EDuration="5.196666588s" totalImagesPullingTime="3.992842753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:11:53.194942459 +0000 UTC m=+451.753025044" watchObservedRunningTime="2026-08-03 16:11:53.196666588 +0000 UTC m=+451.754749172" Aug 03 16:11:55.178224 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.178188 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" event={"ID":"55013419-ac83-48b8-a590-93275df3b798","Type":"ContainerStarted","Data":"47ee2dba4a5927b931e99961ae2fca91fd903f398a3ecf86e983352a6f86dc81"} Aug 03 16:11:55.178649 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.178340 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" Aug 03 16:11:55.202070 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.202027 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" podStartSLOduration=2.078562801 podStartE2EDuration="5.202011814s" totalImagesPullingTime="3.123449013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:11:55.200601237 +0000 UTC m=+453.758683844" watchObservedRunningTime="2026-08-03 16:11:55.202011814 +0000 UTC m=+453.760094395" Aug 03 16:11:55.536467 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.536435 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-x8fwc"] Aug 03 16:11:55.540063 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.540039 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" Aug 03 16:11:55.543672 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.543649 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vzpw6\"" Aug 03 16:11:55.553147 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.553113 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-x8fwc"] Aug 03 16:11:55.662282 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.662246 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsc7v\" (UniqueName: \"kubernetes.io/projected/0f459b6a-8dfc-4626-9ee2-d5d025bc3a37-kube-api-access-fsc7v\") pod \"authorino-operator-68d88948d8-x8fwc\" (UID: \"0f459b6a-8dfc-4626-9ee2-d5d025bc3a37\") " pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" Aug 03 16:11:55.763267 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.763229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsc7v\" (UniqueName: \"kubernetes.io/projected/0f459b6a-8dfc-4626-9ee2-d5d025bc3a37-kube-api-access-fsc7v\") pod \"authorino-operator-68d88948d8-x8fwc\" (UID: \"0f459b6a-8dfc-4626-9ee2-d5d025bc3a37\") " pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" Aug 03 16:11:55.775957 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.775922 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsc7v\" (UniqueName: \"kubernetes.io/projected/0f459b6a-8dfc-4626-9ee2-d5d025bc3a37-kube-api-access-fsc7v\") pod \"authorino-operator-68d88948d8-x8fwc\" (UID: \"0f459b6a-8dfc-4626-9ee2-d5d025bc3a37\") " pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" Aug 03 16:11:55.850294 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:55.850205 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" Aug 03 16:11:56.015775 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:56.015753 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-x8fwc"] Aug 03 16:11:56.018225 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:11:56.018198 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f459b6a_8dfc_4626_9ee2_d5d025bc3a37.slice/crio-b3fd2e0956a95a40188b7eda8f9ecaac923138cdb605d39d7132512a0ba82d40 WatchSource:0}: Error finding container b3fd2e0956a95a40188b7eda8f9ecaac923138cdb605d39d7132512a0ba82d40: Status 404 returned error can't find the container with id b3fd2e0956a95a40188b7eda8f9ecaac923138cdb605d39d7132512a0ba82d40 Aug 03 16:11:56.185193 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:56.185110 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" event={"ID":"0f459b6a-8dfc-4626-9ee2-d5d025bc3a37","Type":"ContainerStarted","Data":"b3fd2e0956a95a40188b7eda8f9ecaac923138cdb605d39d7132512a0ba82d40"} Aug 03 16:11:58.197516 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:58.197475 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" event={"ID":"0f459b6a-8dfc-4626-9ee2-d5d025bc3a37","Type":"ContainerStarted","Data":"8c8b6e38c5777b6c7a92ed79251cbb81e59387ad3b1b06c9aeeb2828c50dc3fb"} Aug 03 16:11:58.197994 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:58.197608 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" Aug 03 16:11:58.216338 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:11:58.216293 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" podStartSLOduration=1.67026313 podStartE2EDuration="3.216278849s" totalImagesPullingTime="1.546015719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:11:58.215526994 +0000 UTC m=+456.773609576" watchObservedRunningTime="2026-08-03 16:11:58.216278849 +0000 UTC m=+456.774361433" Aug 03 16:12:04.172667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:04.172639 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:12:05.520347 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.520317 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz"] Aug 03 16:12:05.523824 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.523808 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:05.542062 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.542036 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz"] Aug 03 16:12:05.655855 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.655817 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxjqj\" (UniqueName: \"kubernetes.io/projected/dad49225-cd16-4ec8-a8d9-4a223c641479-kube-api-access-qxjqj\") pod \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" (UID: \"dad49225-cd16-4ec8-a8d9-4a223c641479\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:05.656030 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.655882 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dad49225-cd16-4ec8-a8d9-4a223c641479-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" (UID: \"dad49225-cd16-4ec8-a8d9-4a223c641479\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:05.757312 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.757274 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dad49225-cd16-4ec8-a8d9-4a223c641479-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" (UID: \"dad49225-cd16-4ec8-a8d9-4a223c641479\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:05.757478 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.757346 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjqj\" (UniqueName: \"kubernetes.io/projected/dad49225-cd16-4ec8-a8d9-4a223c641479-kube-api-access-qxjqj\") pod \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" (UID: \"dad49225-cd16-4ec8-a8d9-4a223c641479\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:05.757678 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.757659 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dad49225-cd16-4ec8-a8d9-4a223c641479-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" (UID: \"dad49225-cd16-4ec8-a8d9-4a223c641479\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:05.773524 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.773457 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxjqj\" (UniqueName: \"kubernetes.io/projected/dad49225-cd16-4ec8-a8d9-4a223c641479-kube-api-access-qxjqj\") pod \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" (UID: \"dad49225-cd16-4ec8-a8d9-4a223c641479\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:05.834324 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.834283 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:05.954223 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.954189 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj"] Aug 03 16:12:05.954437 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.954410 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" podUID="26bd43fb-5229-410c-adb6-929f41fd554f" containerName="manager" containerID="cri-o://cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1" gracePeriod=2 Aug 03 16:12:05.965020 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.964677 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj"] Aug 03 16:12:05.989059 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.989033 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz"] Aug 03 16:12:05.997854 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.997832 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht"] Aug 03 16:12:05.998205 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.998191 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26bd43fb-5229-410c-adb6-929f41fd554f" containerName="manager" Aug 03 16:12:05.998205 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.998206 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="26bd43fb-5229-410c-adb6-929f41fd554f" containerName="manager" Aug 03 16:12:05.998311 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:05.998261 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="26bd43fb-5229-410c-adb6-929f41fd554f" containerName="manager" Aug 03 16:12:06.001362 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.001341 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz"] Aug 03 16:12:06.001470 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.001432 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:06.007573 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.007551 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht"] Aug 03 16:12:06.018556 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.018513 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7"] Aug 03 16:12:06.022877 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.022849 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:06.041562 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.041386 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7"] Aug 03 16:12:06.118353 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:06.118311 2582 log.go:32] "RunPodSandbox from runtime service failed" err=< Aug 03 16:12:06.118353 ip-10-0-142-140 kubenswrapper[2582]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6bd956ccf-8vmtz_kuadrant-system_dad49225-cd16-4ec8-a8d9-4a223c641479_0(4f0c483d0b62d9e2bb8be3b9d940ace418e06c0548e09d5f0e836bf216dd681b): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6bd956ccf-8vmtz to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"4f0c483d0b62d9e2bb8be3b9d940ace418e06c0548e09d5f0e836bf216dd681b" Netns:"/var/run/netns/5dff38b9-9649-4aef-9fc5-8658ba0dccae" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-6bd956ccf-8vmtz;K8S_POD_INFRA_CONTAINER_ID=4f0c483d0b62d9e2bb8be3b9d940ace418e06c0548e09d5f0e836bf216dd681b;K8S_POD_UID=dad49225-cd16-4ec8-a8d9-4a223c641479" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz/dad49225-cd16-4ec8-a8d9-4a223c641479]: error setting the networks status: SetPodNetworkStatusAnnotation: failed to update the pod kuadrant-operator-controller-manager-6bd956ccf-8vmtz in out of cluster comm: SetNetworkStatus: failed to update the pod kuadrant-operator-controller-manager-6bd956ccf-8vmtz in out of cluster comm: status update failed for pod kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz: pods "kuadrant-operator-controller-manager-6bd956ccf-8vmtz" not found Aug 03 16:12:06.118353 ip-10-0-142-140 kubenswrapper[2582]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Aug 03 16:12:06.118353 ip-10-0-142-140 kubenswrapper[2582]: > Aug 03 16:12:06.118559 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:06.118387 2582 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Aug 03 16:12:06.118559 ip-10-0-142-140 kubenswrapper[2582]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6bd956ccf-8vmtz_kuadrant-system_dad49225-cd16-4ec8-a8d9-4a223c641479_0(4f0c483d0b62d9e2bb8be3b9d940ace418e06c0548e09d5f0e836bf216dd681b): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6bd956ccf-8vmtz to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"4f0c483d0b62d9e2bb8be3b9d940ace418e06c0548e09d5f0e836bf216dd681b" Netns:"/var/run/netns/5dff38b9-9649-4aef-9fc5-8658ba0dccae" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-6bd956ccf-8vmtz;K8S_POD_INFRA_CONTAINER_ID=4f0c483d0b62d9e2bb8be3b9d940ace418e06c0548e09d5f0e836bf216dd681b;K8S_POD_UID=dad49225-cd16-4ec8-a8d9-4a223c641479" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz/dad49225-cd16-4ec8-a8d9-4a223c641479]: error setting the networks status: SetPodNetworkStatusAnnotation: failed to update the pod kuadrant-operator-controller-manager-6bd956ccf-8vmtz in out of cluster comm: SetNetworkStatus: failed to update the pod kuadrant-operator-controller-manager-6bd956ccf-8vmtz in out of cluster comm: status update failed for pod kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz: pods "kuadrant-operator-controller-manager-6bd956ccf-8vmtz" not found Aug 03 16:12:06.118559 ip-10-0-142-140 kubenswrapper[2582]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Aug 03 16:12:06.118559 ip-10-0-142-140 kubenswrapper[2582]: > pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:06.162249 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.162209 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvd88\" (UniqueName: \"kubernetes.io/projected/a534f156-3e00-4f74-b1b6-150f7cca14ed-kube-api-access-dvd88\") pod \"kuadrant-operator-controller-manager-58fc56757c-b2mht\" (UID: \"a534f156-3e00-4f74-b1b6-150f7cca14ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:06.162387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.162326 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm9ct\" (UniqueName: \"kubernetes.io/projected/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-kube-api-access-wm9ct\") pod \"kuadrant-operator-controller-manager-6bd956ccf-wksn7\" (UID: \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:06.162447 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.162387 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a534f156-3e00-4f74-b1b6-150f7cca14ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-b2mht\" (UID: \"a534f156-3e00-4f74-b1b6-150f7cca14ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:06.162447 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.162422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-wksn7\" (UID: \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:06.187296 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.187271 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:12:06.188004 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.187987 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-r8g86" Aug 03 16:12:06.240998 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.240969 2582 generic.go:356] "Generic (PLEG): container finished" podID="26bd43fb-5229-410c-adb6-929f41fd554f" containerID="cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1" exitCode=0 Aug 03 16:12:06.241190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.241009 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-qpprj" Aug 03 16:12:06.241190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.241025 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:06.241190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.241075 2582 scope.go:117] "RemoveContainer" containerID="cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1" Aug 03 16:12:06.244172 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:06.244142 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" is forbidden: User \"system:node:ip-10-0-142-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-140.ec2.internal' and this object" podUID="dad49225-cd16-4ec8-a8d9-4a223c641479" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:06.246657 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:06.246608 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" is forbidden: User \"system:node:ip-10-0-142-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-140.ec2.internal' and this object" podUID="dad49225-cd16-4ec8-a8d9-4a223c641479" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:06.248953 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.248938 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:06.251508 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:06.251397 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" is forbidden: User \"system:node:ip-10-0-142-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-140.ec2.internal' and this object" podUID="dad49225-cd16-4ec8-a8d9-4a223c641479" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:06.251594 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.251518 2582 scope.go:117] "RemoveContainer" containerID="cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1" Aug 03 16:12:06.251882 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:06.251861 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1\": container with ID starting with cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1 not found: ID does not exist" containerID="cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1" Aug 03 16:12:06.251963 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.251887 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1"} err="failed to get container status \"cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1\": rpc error: code = NotFound desc = could not find container \"cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1\": container with ID starting with cf42dc136c17c97e6c4416f233b181a911bbab5c2c6727c05df9c2e0fac62af1 not found: ID does not exist" Aug 03 16:12:06.262919 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.262894 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bd43fb-5229-410c-adb6-929f41fd554f-extensions-socket-volume\") pod \"26bd43fb-5229-410c-adb6-929f41fd554f\" (UID: \"26bd43fb-5229-410c-adb6-929f41fd554f\") " Aug 03 16:12:06.263073 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.262939 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ppj7\" (UniqueName: \"kubernetes.io/projected/26bd43fb-5229-410c-adb6-929f41fd554f-kube-api-access-7ppj7\") pod \"26bd43fb-5229-410c-adb6-929f41fd554f\" (UID: \"26bd43fb-5229-410c-adb6-929f41fd554f\") " Aug 03 16:12:06.263073 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.263057 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvd88\" (UniqueName: \"kubernetes.io/projected/a534f156-3e00-4f74-b1b6-150f7cca14ed-kube-api-access-dvd88\") pod \"kuadrant-operator-controller-manager-58fc56757c-b2mht\" (UID: \"a534f156-3e00-4f74-b1b6-150f7cca14ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:06.263184 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.263157 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm9ct\" (UniqueName: \"kubernetes.io/projected/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-kube-api-access-wm9ct\") pod \"kuadrant-operator-controller-manager-6bd956ccf-wksn7\" (UID: \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:06.263232 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.263216 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a534f156-3e00-4f74-b1b6-150f7cca14ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-b2mht\" (UID: \"a534f156-3e00-4f74-b1b6-150f7cca14ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:06.263282 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.263247 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-wksn7\" (UID: \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:06.263453 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.263380 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26bd43fb-5229-410c-adb6-929f41fd554f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "26bd43fb-5229-410c-adb6-929f41fd554f" (UID: "26bd43fb-5229-410c-adb6-929f41fd554f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:12:06.263658 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.263607 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bd956ccf-wksn7\" (UID: \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:06.263743 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.263721 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a534f156-3e00-4f74-b1b6-150f7cca14ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-b2mht\" (UID: \"a534f156-3e00-4f74-b1b6-150f7cca14ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:06.265210 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.265186 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26bd43fb-5229-410c-adb6-929f41fd554f-kube-api-access-7ppj7" (OuterVolumeSpecName: "kube-api-access-7ppj7") pod "26bd43fb-5229-410c-adb6-929f41fd554f" (UID: "26bd43fb-5229-410c-adb6-929f41fd554f"). InnerVolumeSpecName "kube-api-access-7ppj7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:12:06.274703 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.274680 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvd88\" (UniqueName: \"kubernetes.io/projected/a534f156-3e00-4f74-b1b6-150f7cca14ed-kube-api-access-dvd88\") pod \"kuadrant-operator-controller-manager-58fc56757c-b2mht\" (UID: \"a534f156-3e00-4f74-b1b6-150f7cca14ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:06.274938 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.274923 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm9ct\" (UniqueName: \"kubernetes.io/projected/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-kube-api-access-wm9ct\") pod \"kuadrant-operator-controller-manager-6bd956ccf-wksn7\" (UID: \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:06.343450 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.343360 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:06.350813 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.350791 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:06.364041 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.364009 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxjqj\" (UniqueName: \"kubernetes.io/projected/dad49225-cd16-4ec8-a8d9-4a223c641479-kube-api-access-qxjqj\") pod \"dad49225-cd16-4ec8-a8d9-4a223c641479\" (UID: \"dad49225-cd16-4ec8-a8d9-4a223c641479\") " Aug 03 16:12:06.364150 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.364064 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dad49225-cd16-4ec8-a8d9-4a223c641479-extensions-socket-volume\") pod \"dad49225-cd16-4ec8-a8d9-4a223c641479\" (UID: \"dad49225-cd16-4ec8-a8d9-4a223c641479\") " Aug 03 16:12:06.364291 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.364274 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26bd43fb-5229-410c-adb6-929f41fd554f-extensions-socket-volume\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:12:06.364345 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.364296 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ppj7\" (UniqueName: \"kubernetes.io/projected/26bd43fb-5229-410c-adb6-929f41fd554f-kube-api-access-7ppj7\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:12:06.364394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.364350 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dad49225-cd16-4ec8-a8d9-4a223c641479-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dad49225-cd16-4ec8-a8d9-4a223c641479" (UID: "dad49225-cd16-4ec8-a8d9-4a223c641479"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:12:06.365998 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.365965 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dad49225-cd16-4ec8-a8d9-4a223c641479-kube-api-access-qxjqj" (OuterVolumeSpecName: "kube-api-access-qxjqj") pod "dad49225-cd16-4ec8-a8d9-4a223c641479" (UID: "dad49225-cd16-4ec8-a8d9-4a223c641479"). InnerVolumeSpecName "kube-api-access-qxjqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:12:06.465322 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.465287 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxjqj\" (UniqueName: \"kubernetes.io/projected/dad49225-cd16-4ec8-a8d9-4a223c641479-kube-api-access-qxjqj\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:12:06.465322 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.465323 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dad49225-cd16-4ec8-a8d9-4a223c641479-extensions-socket-volume\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:12:06.482206 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.482181 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht"] Aug 03 16:12:06.483981 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:12:06.483956 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda534f156_3e00_4f74_b1b6_150f7cca14ed.slice/crio-be4f711c049dbd7f27274bff05809e3a2369b536adb198e717ec64ff39ce48c6 WatchSource:0}: Error finding container be4f711c049dbd7f27274bff05809e3a2369b536adb198e717ec64ff39ce48c6: Status 404 returned error can't find the container with id be4f711c049dbd7f27274bff05809e3a2369b536adb198e717ec64ff39ce48c6 Aug 03 16:12:06.508285 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:06.508261 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7"] Aug 03 16:12:06.517737 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:12:06.517713 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07eccf18_8fb2_4dd2_9b34_e43c9b60d464.slice/crio-9c33dc3f7190f40be4123c38a1d9a6594f91d9136b25caf8248658a5b8dfd918 WatchSource:0}: Error finding container 9c33dc3f7190f40be4123c38a1d9a6594f91d9136b25caf8248658a5b8dfd918: Status 404 returned error can't find the container with id 9c33dc3f7190f40be4123c38a1d9a6594f91d9136b25caf8248658a5b8dfd918 Aug 03 16:12:06.554493 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:06.554460 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" is forbidden: User \"system:node:ip-10-0-142-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-140.ec2.internal' and this object" podUID="dad49225-cd16-4ec8-a8d9-4a223c641479" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:07.248392 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.248303 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" event={"ID":"a534f156-3e00-4f74-b1b6-150f7cca14ed","Type":"ContainerStarted","Data":"9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55"} Aug 03 16:12:07.248392 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.248358 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:07.248392 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.248380 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" event={"ID":"a534f156-3e00-4f74-b1b6-150f7cca14ed","Type":"ContainerStarted","Data":"be4f711c049dbd7f27274bff05809e3a2369b536adb198e717ec64ff39ce48c6"} Aug 03 16:12:07.250933 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.250909 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" event={"ID":"07eccf18-8fb2-4dd2-9b34-e43c9b60d464","Type":"ContainerStarted","Data":"e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25"} Aug 03 16:12:07.251033 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.250952 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" event={"ID":"07eccf18-8fb2-4dd2-9b34-e43c9b60d464","Type":"ContainerStarted","Data":"9c33dc3f7190f40be4123c38a1d9a6594f91d9136b25caf8248658a5b8dfd918"} Aug 03 16:12:07.251033 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.250986 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:07.251033 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.251007 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:07.257062 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:07.257035 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" is forbidden: User \"system:node:ip-10-0-142-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-140.ec2.internal' and this object" podUID="dad49225-cd16-4ec8-a8d9-4a223c641479" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:07.294823 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.294774 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" podStartSLOduration=2.2947598940000002 podStartE2EDuration="2.294759894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:07.293303942 +0000 UTC m=+465.851386527" watchObservedRunningTime="2026-08-03 16:12:07.294759894 +0000 UTC m=+465.852842479" Aug 03 16:12:07.324027 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:07.323989 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6bd956ccf-8vmtz\" is forbidden: User \"system:node:ip-10-0-142-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-140.ec2.internal' and this object" podUID="dad49225-cd16-4ec8-a8d9-4a223c641479" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-8vmtz" Aug 03 16:12:07.325013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.324975 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" podStartSLOduration=2.324962707 podStartE2EDuration="2.324962707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:07.321471135 +0000 UTC m=+465.879553720" watchObservedRunningTime="2026-08-03 16:12:07.324962707 +0000 UTC m=+465.883045292" Aug 03 16:12:07.995120 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.995085 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26bd43fb-5229-410c-adb6-929f41fd554f" path="/var/lib/kubelet/pods/26bd43fb-5229-410c-adb6-929f41fd554f/volumes" Aug 03 16:12:07.995483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:07.995416 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dad49225-cd16-4ec8-a8d9-4a223c641479" path="/var/lib/kubelet/pods/dad49225-cd16-4ec8-a8d9-4a223c641479/volumes" Aug 03 16:12:09.205332 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:09.205299 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-x8fwc" Aug 03 16:12:18.257970 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.257942 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:18.258354 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.258009 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:18.343096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.343066 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht"] Aug 03 16:12:18.343307 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.343281 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" podUID="a534f156-3e00-4f74-b1b6-150f7cca14ed" containerName="manager" containerID="cri-o://9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55" gracePeriod=10 Aug 03 16:12:18.579984 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.579959 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:18.669682 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.669651 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvd88\" (UniqueName: \"kubernetes.io/projected/a534f156-3e00-4f74-b1b6-150f7cca14ed-kube-api-access-dvd88\") pod \"a534f156-3e00-4f74-b1b6-150f7cca14ed\" (UID: \"a534f156-3e00-4f74-b1b6-150f7cca14ed\") " Aug 03 16:12:18.669860 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.669729 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a534f156-3e00-4f74-b1b6-150f7cca14ed-extensions-socket-volume\") pod \"a534f156-3e00-4f74-b1b6-150f7cca14ed\" (UID: \"a534f156-3e00-4f74-b1b6-150f7cca14ed\") " Aug 03 16:12:18.670131 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.670107 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a534f156-3e00-4f74-b1b6-150f7cca14ed-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a534f156-3e00-4f74-b1b6-150f7cca14ed" (UID: "a534f156-3e00-4f74-b1b6-150f7cca14ed"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:12:18.671667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.671635 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a534f156-3e00-4f74-b1b6-150f7cca14ed-kube-api-access-dvd88" (OuterVolumeSpecName: "kube-api-access-dvd88") pod "a534f156-3e00-4f74-b1b6-150f7cca14ed" (UID: "a534f156-3e00-4f74-b1b6-150f7cca14ed"). InnerVolumeSpecName "kube-api-access-dvd88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:12:18.730611 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.730580 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s"] Aug 03 16:12:18.731037 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.731019 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a534f156-3e00-4f74-b1b6-150f7cca14ed" containerName="manager" Aug 03 16:12:18.731121 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.731040 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a534f156-3e00-4f74-b1b6-150f7cca14ed" containerName="manager" Aug 03 16:12:18.731174 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.731142 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a534f156-3e00-4f74-b1b6-150f7cca14ed" containerName="manager" Aug 03 16:12:18.734104 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.734086 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:18.757697 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.757664 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s"] Aug 03 16:12:18.770400 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.770374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7298fbb9-d217-4c0a-9289-1dbeab889ded-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-t5d4s\" (UID: \"7298fbb9-d217-4c0a-9289-1dbeab889ded\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:18.770504 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.770412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56hrg\" (UniqueName: \"kubernetes.io/projected/7298fbb9-d217-4c0a-9289-1dbeab889ded-kube-api-access-56hrg\") pod \"kuadrant-operator-controller-manager-68cf589589-t5d4s\" (UID: \"7298fbb9-d217-4c0a-9289-1dbeab889ded\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:18.770504 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.770442 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a534f156-3e00-4f74-b1b6-150f7cca14ed-extensions-socket-volume\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:12:18.770504 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.770452 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvd88\" (UniqueName: \"kubernetes.io/projected/a534f156-3e00-4f74-b1b6-150f7cca14ed-kube-api-access-dvd88\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:12:18.871400 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.871315 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7298fbb9-d217-4c0a-9289-1dbeab889ded-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-t5d4s\" (UID: \"7298fbb9-d217-4c0a-9289-1dbeab889ded\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:18.871400 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.871366 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56hrg\" (UniqueName: \"kubernetes.io/projected/7298fbb9-d217-4c0a-9289-1dbeab889ded-kube-api-access-56hrg\") pod \"kuadrant-operator-controller-manager-68cf589589-t5d4s\" (UID: \"7298fbb9-d217-4c0a-9289-1dbeab889ded\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:18.871758 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.871737 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7298fbb9-d217-4c0a-9289-1dbeab889ded-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-t5d4s\" (UID: \"7298fbb9-d217-4c0a-9289-1dbeab889ded\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:18.882235 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:18.882207 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56hrg\" (UniqueName: \"kubernetes.io/projected/7298fbb9-d217-4c0a-9289-1dbeab889ded-kube-api-access-56hrg\") pod \"kuadrant-operator-controller-manager-68cf589589-t5d4s\" (UID: \"7298fbb9-d217-4c0a-9289-1dbeab889ded\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:19.045329 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.045291 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:19.181771 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.181745 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s"] Aug 03 16:12:19.182909 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:12:19.182888 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7298fbb9_d217_4c0a_9289_1dbeab889ded.slice/crio-71a048850307cc8f472130db9e4a50bb0f55f1bdf00153d817145c77fc19bf41 WatchSource:0}: Error finding container 71a048850307cc8f472130db9e4a50bb0f55f1bdf00153d817145c77fc19bf41: Status 404 returned error can't find the container with id 71a048850307cc8f472130db9e4a50bb0f55f1bdf00153d817145c77fc19bf41 Aug 03 16:12:19.314895 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.314866 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" event={"ID":"7298fbb9-d217-4c0a-9289-1dbeab889ded","Type":"ContainerStarted","Data":"4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae"} Aug 03 16:12:19.315269 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.314912 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" event={"ID":"7298fbb9-d217-4c0a-9289-1dbeab889ded","Type":"ContainerStarted","Data":"71a048850307cc8f472130db9e4a50bb0f55f1bdf00153d817145c77fc19bf41"} Aug 03 16:12:19.315269 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.314955 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:19.316227 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.316206 2582 generic.go:356] "Generic (PLEG): container finished" podID="a534f156-3e00-4f74-b1b6-150f7cca14ed" containerID="9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55" exitCode=0 Aug 03 16:12:19.316336 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.316269 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" Aug 03 16:12:19.316336 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.316284 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" event={"ID":"a534f156-3e00-4f74-b1b6-150f7cca14ed","Type":"ContainerDied","Data":"9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55"} Aug 03 16:12:19.316336 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.316322 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht" event={"ID":"a534f156-3e00-4f74-b1b6-150f7cca14ed","Type":"ContainerDied","Data":"be4f711c049dbd7f27274bff05809e3a2369b536adb198e717ec64ff39ce48c6"} Aug 03 16:12:19.316485 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.316344 2582 scope.go:117] "RemoveContainer" containerID="9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55" Aug 03 16:12:19.324647 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.324630 2582 scope.go:117] "RemoveContainer" containerID="9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55" Aug 03 16:12:19.324893 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:19.324873 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55\": container with ID starting with 9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55 not found: ID does not exist" containerID="9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55" Aug 03 16:12:19.324961 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.324900 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55"} err="failed to get container status \"9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55\": rpc error: code = NotFound desc = could not find container \"9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55\": container with ID starting with 9a54196c10a9c660dc2603572057be485697f42583d3cdbb3c1c3431fed42d55 not found: ID does not exist" Aug 03 16:12:19.342268 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.342214 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" podStartSLOduration=1.3421946710000001 podStartE2EDuration="1.342194671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:19.341265053 +0000 UTC m=+477.899347640" watchObservedRunningTime="2026-08-03 16:12:19.342194671 +0000 UTC m=+477.900277257" Aug 03 16:12:19.359017 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.358994 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht"] Aug 03 16:12:19.367119 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.367098 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-b2mht"] Aug 03 16:12:19.998902 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:19.998871 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a534f156-3e00-4f74-b1b6-150f7cca14ed" path="/var/lib/kubelet/pods/a534f156-3e00-4f74-b1b6-150f7cca14ed/volumes" Aug 03 16:12:30.325002 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.324970 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:12:30.387060 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.387026 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7"] Aug 03 16:12:30.387337 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.387261 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" podUID="07eccf18-8fb2-4dd2-9b34-e43c9b60d464" containerName="manager" containerID="cri-o://e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25" gracePeriod=10 Aug 03 16:12:30.625711 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.625690 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:30.667653 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.667601 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm9ct\" (UniqueName: \"kubernetes.io/projected/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-kube-api-access-wm9ct\") pod \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\" (UID: \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\") " Aug 03 16:12:30.667805 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.667690 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-extensions-socket-volume\") pod \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\" (UID: \"07eccf18-8fb2-4dd2-9b34-e43c9b60d464\") " Aug 03 16:12:30.668059 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.668032 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "07eccf18-8fb2-4dd2-9b34-e43c9b60d464" (UID: "07eccf18-8fb2-4dd2-9b34-e43c9b60d464"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:12:30.669842 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.669818 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-kube-api-access-wm9ct" (OuterVolumeSpecName: "kube-api-access-wm9ct") pod "07eccf18-8fb2-4dd2-9b34-e43c9b60d464" (UID: "07eccf18-8fb2-4dd2-9b34-e43c9b60d464"). InnerVolumeSpecName "kube-api-access-wm9ct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:12:30.768474 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.768424 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-extensions-socket-volume\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:12:30.768474 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:30.768466 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm9ct\" (UniqueName: \"kubernetes.io/projected/07eccf18-8fb2-4dd2-9b34-e43c9b60d464-kube-api-access-wm9ct\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:12:31.379991 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.379959 2582 generic.go:356] "Generic (PLEG): container finished" podID="07eccf18-8fb2-4dd2-9b34-e43c9b60d464" containerID="e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25" exitCode=0 Aug 03 16:12:31.380403 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.380002 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" Aug 03 16:12:31.380403 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.380045 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" event={"ID":"07eccf18-8fb2-4dd2-9b34-e43c9b60d464","Type":"ContainerDied","Data":"e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25"} Aug 03 16:12:31.380403 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.380094 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7" event={"ID":"07eccf18-8fb2-4dd2-9b34-e43c9b60d464","Type":"ContainerDied","Data":"9c33dc3f7190f40be4123c38a1d9a6594f91d9136b25caf8248658a5b8dfd918"} Aug 03 16:12:31.380403 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.380116 2582 scope.go:117] "RemoveContainer" containerID="e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25" Aug 03 16:12:31.388806 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.388565 2582 scope.go:117] "RemoveContainer" containerID="e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25" Aug 03 16:12:31.388861 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:31.388848 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25\": container with ID starting with e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25 not found: ID does not exist" containerID="e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25" Aug 03 16:12:31.388903 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.388872 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25"} err="failed to get container status \"e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25\": rpc error: code = NotFound desc = could not find container \"e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25\": container with ID starting with e4f37b844760640774ac7afbc227c02ca40d410c48e7d3da05b3fcd44ab2aa25 not found: ID does not exist" Aug 03 16:12:31.404352 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.404333 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7"] Aug 03 16:12:31.408232 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.408213 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bd956ccf-wksn7"] Aug 03 16:12:31.995704 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:31.995671 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07eccf18-8fb2-4dd2-9b34-e43c9b60d464" path="/var/lib/kubelet/pods/07eccf18-8fb2-4dd2-9b34-e43c9b60d464/volumes" Aug 03 16:12:34.747399 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.747371 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn"] Aug 03 16:12:34.747913 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.747897 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07eccf18-8fb2-4dd2-9b34-e43c9b60d464" containerName="manager" Aug 03 16:12:34.747969 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.747915 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="07eccf18-8fb2-4dd2-9b34-e43c9b60d464" containerName="manager" Aug 03 16:12:34.748005 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.747992 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="07eccf18-8fb2-4dd2-9b34-e43c9b60d464" containerName="manager" Aug 03 16:12:34.750797 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.750783 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.754284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.754266 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gnf6x\"" Aug 03 16:12:34.762885 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.762861 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn"] Aug 03 16:12:34.800163 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.800135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbd9z\" (UniqueName: \"kubernetes.io/projected/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-kube-api-access-wbd9z\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.800309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.800175 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.800309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.800213 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.800309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.800237 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.800309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.800253 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.800309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.800277 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.800552 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.800390 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.800552 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.800433 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.800552 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.800464 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901132 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901132 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901149 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901179 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901210 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901324 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbd9z\" (UniqueName: \"kubernetes.io/projected/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-kube-api-access-wbd9z\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901680 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901434 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901680 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901794 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901698 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901794 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901754 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901914 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901807 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.901914 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.901856 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.902029 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.902005 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.903697 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.903667 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.903825 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.903809 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.909751 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.909732 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:34.909981 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:34.909966 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbd9z\" (UniqueName: \"kubernetes.io/projected/8db2abdc-9a5a-49a2-9df8-909dacd7cf61-kube-api-access-wbd9z\") pod \"maas-default-gateway-openshift-default-78bb78ff78-4htkn\" (UID: \"8db2abdc-9a5a-49a2-9df8-909dacd7cf61\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:35.061370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:35.061329 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:35.197038 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:35.197013 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn"] Aug 03 16:12:35.197899 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:12:35.197874 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db2abdc_9a5a_49a2_9df8_909dacd7cf61.slice/crio-871e8488c6053bae5898cea0be4cc977fc8f28ca52f746b1293968b9262e8b94 WatchSource:0}: Error finding container 871e8488c6053bae5898cea0be4cc977fc8f28ca52f746b1293968b9262e8b94: Status 404 returned error can't find the container with id 871e8488c6053bae5898cea0be4cc977fc8f28ca52f746b1293968b9262e8b94 Aug 03 16:12:35.199920 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:35.199892 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:12:35.199989 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:35.199954 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:12:35.199989 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:35.199980 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:12:35.404469 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:35.404380 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" event={"ID":"8db2abdc-9a5a-49a2-9df8-909dacd7cf61","Type":"ContainerStarted","Data":"8f3204fd519298b93de317a8bc1c852a5bf43ff97d576fca7b95db4b0a835363"} Aug 03 16:12:35.404469 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:35.404438 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" event={"ID":"8db2abdc-9a5a-49a2-9df8-909dacd7cf61","Type":"ContainerStarted","Data":"871e8488c6053bae5898cea0be4cc977fc8f28ca52f746b1293968b9262e8b94"} Aug 03 16:12:35.425795 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:35.425748 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" podStartSLOduration=1.425732053 podStartE2EDuration="1.425732053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:35.423396575 +0000 UTC m=+493.981479161" watchObservedRunningTime="2026-08-03 16:12:35.425732053 +0000 UTC m=+493.983814637" Aug 03 16:12:36.062437 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:36.062408 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:36.067077 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:36.067056 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:36.411319 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:36.411236 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:36.412309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:36.412291 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:12:47.528083 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.528049 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-fnf2m"] Aug 03 16:12:47.530659 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.530641 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fnf2m" Aug 03 16:12:47.533832 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.533815 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xfrgp\"" Aug 03 16:12:47.539445 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.539414 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fnf2m"] Aug 03 16:12:47.612532 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.612498 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d52mb\" (UniqueName: \"kubernetes.io/projected/ceacd810-5fe5-494d-867e-6a764422c87d-kube-api-access-d52mb\") pod \"authorino-b54df94b4-fnf2m\" (UID: \"ceacd810-5fe5-494d-867e-6a764422c87d\") " pod="kuadrant-system/authorino-b54df94b4-fnf2m" Aug 03 16:12:47.661570 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.661539 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d78rq"] Aug 03 16:12:47.664175 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.664157 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-d78rq" Aug 03 16:12:47.672252 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.672231 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d78rq"] Aug 03 16:12:47.713991 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.713958 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d52mb\" (UniqueName: \"kubernetes.io/projected/ceacd810-5fe5-494d-867e-6a764422c87d-kube-api-access-d52mb\") pod \"authorino-b54df94b4-fnf2m\" (UID: \"ceacd810-5fe5-494d-867e-6a764422c87d\") " pod="kuadrant-system/authorino-b54df94b4-fnf2m" Aug 03 16:12:47.714159 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.714010 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h2c6\" (UniqueName: \"kubernetes.io/projected/6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d-kube-api-access-8h2c6\") pod \"authorino-c548b5d7d-d78rq\" (UID: \"6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d\") " pod="kuadrant-system/authorino-c548b5d7d-d78rq" Aug 03 16:12:47.725429 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.725409 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d52mb\" (UniqueName: \"kubernetes.io/projected/ceacd810-5fe5-494d-867e-6a764422c87d-kube-api-access-d52mb\") pod \"authorino-b54df94b4-fnf2m\" (UID: \"ceacd810-5fe5-494d-867e-6a764422c87d\") " pod="kuadrant-system/authorino-b54df94b4-fnf2m" Aug 03 16:12:47.815082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.814995 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h2c6\" (UniqueName: \"kubernetes.io/projected/6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d-kube-api-access-8h2c6\") pod \"authorino-c548b5d7d-d78rq\" (UID: \"6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d\") " pod="kuadrant-system/authorino-c548b5d7d-d78rq" Aug 03 16:12:47.823953 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.823923 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h2c6\" (UniqueName: \"kubernetes.io/projected/6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d-kube-api-access-8h2c6\") pod \"authorino-c548b5d7d-d78rq\" (UID: \"6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d\") " pod="kuadrant-system/authorino-c548b5d7d-d78rq" Aug 03 16:12:47.840873 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.840846 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fnf2m" Aug 03 16:12:47.973177 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.973139 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-d78rq" Aug 03 16:12:47.973891 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:47.973862 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fnf2m"] Aug 03 16:12:47.976341 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:12:47.976314 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podceacd810_5fe5_494d_867e_6a764422c87d.slice/crio-0199154f64fba68331f124975c85d7e89f7f28c88ade01f0966011d922586659 WatchSource:0}: Error finding container 0199154f64fba68331f124975c85d7e89f7f28c88ade01f0966011d922586659: Status 404 returned error can't find the container with id 0199154f64fba68331f124975c85d7e89f7f28c88ade01f0966011d922586659 Aug 03 16:12:48.099709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:48.099683 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d78rq"] Aug 03 16:12:48.100760 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:12:48.100732 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e0601f5_f5c3_44e3_bfd1_77e1956e5c8d.slice/crio-6c539b190d41b5acfde27b68d3a492b10995efed97654e01cbeee93433cf211d WatchSource:0}: Error finding container 6c539b190d41b5acfde27b68d3a492b10995efed97654e01cbeee93433cf211d: Status 404 returned error can't find the container with id 6c539b190d41b5acfde27b68d3a492b10995efed97654e01cbeee93433cf211d Aug 03 16:12:48.480449 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:48.480354 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-d78rq" event={"ID":"6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d","Type":"ContainerStarted","Data":"6c539b190d41b5acfde27b68d3a492b10995efed97654e01cbeee93433cf211d"} Aug 03 16:12:48.481419 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:48.481394 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fnf2m" event={"ID":"ceacd810-5fe5-494d-867e-6a764422c87d","Type":"ContainerStarted","Data":"0199154f64fba68331f124975c85d7e89f7f28c88ade01f0966011d922586659"} Aug 03 16:12:50.500287 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:50.500185 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-d78rq" event={"ID":"6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d","Type":"ContainerStarted","Data":"1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060"} Aug 03 16:12:50.501560 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:50.501535 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fnf2m" event={"ID":"ceacd810-5fe5-494d-867e-6a764422c87d","Type":"ContainerStarted","Data":"6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b"} Aug 03 16:12:50.518276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:50.518234 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-d78rq" podStartSLOduration=1.527628099 podStartE2EDuration="3.518220557s" totalImagesPullingTime="1.990592458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:50.516032762 +0000 UTC m=+509.074115348" watchObservedRunningTime="2026-08-03 16:12:50.518220557 +0000 UTC m=+509.076303141" Aug 03 16:12:50.532155 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:50.532103 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-fnf2m" podStartSLOduration=1.427416593 podStartE2EDuration="3.532084633s" totalImagesPullingTime="2.10466804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:12:50.530430489 +0000 UTC m=+509.088513078" watchObservedRunningTime="2026-08-03 16:12:50.532084633 +0000 UTC m=+509.090167217" Aug 03 16:12:50.555849 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:50.555815 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fnf2m"] Aug 03 16:12:52.511798 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:52.511758 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-fnf2m" podUID="ceacd810-5fe5-494d-867e-6a764422c87d" containerName="authorino" containerID="cri-o://6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b" gracePeriod=30 Aug 03 16:12:52.747684 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:52.747661 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fnf2m" Aug 03 16:12:52.861510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:52.861480 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d52mb\" (UniqueName: \"kubernetes.io/projected/ceacd810-5fe5-494d-867e-6a764422c87d-kube-api-access-d52mb\") pod \"ceacd810-5fe5-494d-867e-6a764422c87d\" (UID: \"ceacd810-5fe5-494d-867e-6a764422c87d\") " Aug 03 16:12:52.863514 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:52.863487 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceacd810-5fe5-494d-867e-6a764422c87d-kube-api-access-d52mb" (OuterVolumeSpecName: "kube-api-access-d52mb") pod "ceacd810-5fe5-494d-867e-6a764422c87d" (UID: "ceacd810-5fe5-494d-867e-6a764422c87d"). InnerVolumeSpecName "kube-api-access-d52mb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:12:52.962597 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:52.962559 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d52mb\" (UniqueName: \"kubernetes.io/projected/ceacd810-5fe5-494d-867e-6a764422c87d-kube-api-access-d52mb\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:12:53.517881 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.517844 2582 generic.go:356] "Generic (PLEG): container finished" podID="ceacd810-5fe5-494d-867e-6a764422c87d" containerID="6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b" exitCode=0 Aug 03 16:12:53.518386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.517897 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-fnf2m" Aug 03 16:12:53.518386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.517927 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fnf2m" event={"ID":"ceacd810-5fe5-494d-867e-6a764422c87d","Type":"ContainerDied","Data":"6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b"} Aug 03 16:12:53.518386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.517965 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-fnf2m" event={"ID":"ceacd810-5fe5-494d-867e-6a764422c87d","Type":"ContainerDied","Data":"0199154f64fba68331f124975c85d7e89f7f28c88ade01f0966011d922586659"} Aug 03 16:12:53.518386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.517981 2582 scope.go:117] "RemoveContainer" containerID="6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b" Aug 03 16:12:53.527410 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.527388 2582 scope.go:117] "RemoveContainer" containerID="6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b" Aug 03 16:12:53.527680 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:12:53.527662 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b\": container with ID starting with 6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b not found: ID does not exist" containerID="6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b" Aug 03 16:12:53.527755 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.527694 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b"} err="failed to get container status \"6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b\": rpc error: code = NotFound desc = could not find container \"6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b\": container with ID starting with 6e0ae34d7b9345175bd2efcc1f2015afb8b3447a341f43916063febdb4a75b1b not found: ID does not exist" Aug 03 16:12:53.541034 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.541009 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fnf2m"] Aug 03 16:12:53.544870 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.544851 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-fnf2m"] Aug 03 16:12:53.995265 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:12:53.995185 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceacd810-5fe5-494d-867e-6a764422c87d" path="/var/lib/kubelet/pods/ceacd810-5fe5-494d-867e-6a764422c87d/volumes" Aug 03 16:13:15.167966 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.167883 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xvfmk"] Aug 03 16:13:15.168525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.168414 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceacd810-5fe5-494d-867e-6a764422c87d" containerName="authorino" Aug 03 16:13:15.168525 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.168433 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceacd810-5fe5-494d-867e-6a764422c87d" containerName="authorino" Aug 03 16:13:15.168667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.168528 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceacd810-5fe5-494d-867e-6a764422c87d" containerName="authorino" Aug 03 16:13:15.170761 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.170743 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xvfmk" Aug 03 16:13:15.180030 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.180009 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xvfmk"] Aug 03 16:13:15.360332 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.360297 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q64qp\" (UniqueName: \"kubernetes.io/projected/bb383ff8-0436-4317-9ba7-26820ab05870-kube-api-access-q64qp\") pod \"authorino-5dc7ccb5f-xvfmk\" (UID: \"bb383ff8-0436-4317-9ba7-26820ab05870\") " pod="kuadrant-system/authorino-5dc7ccb5f-xvfmk" Aug 03 16:13:15.461467 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.461383 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q64qp\" (UniqueName: \"kubernetes.io/projected/bb383ff8-0436-4317-9ba7-26820ab05870-kube-api-access-q64qp\") pod \"authorino-5dc7ccb5f-xvfmk\" (UID: \"bb383ff8-0436-4317-9ba7-26820ab05870\") " pod="kuadrant-system/authorino-5dc7ccb5f-xvfmk" Aug 03 16:13:15.465660 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.464066 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xvfmk"] Aug 03 16:13:15.465660 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:13:15.464692 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-q64qp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-xvfmk" podUID="bb383ff8-0436-4317-9ba7-26820ab05870" Aug 03 16:13:15.474263 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.474230 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q64qp\" (UniqueName: \"kubernetes.io/projected/bb383ff8-0436-4317-9ba7-26820ab05870-kube-api-access-q64qp\") pod \"authorino-5dc7ccb5f-xvfmk\" (UID: \"bb383ff8-0436-4317-9ba7-26820ab05870\") " pod="kuadrant-system/authorino-5dc7ccb5f-xvfmk" Aug 03 16:13:15.490743 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.490706 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-588f75498-glhlq"] Aug 03 16:13:15.493390 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.493373 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:13:15.497389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.497367 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Aug 03 16:13:15.503130 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.503102 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-588f75498-glhlq"] Aug 03 16:13:15.637677 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.637648 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xvfmk" Aug 03 16:13:15.643515 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.643492 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xvfmk" Aug 03 16:13:15.663040 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.663006 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v78rf\" (UniqueName: \"kubernetes.io/projected/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-kube-api-access-v78rf\") pod \"authorino-588f75498-glhlq\" (UID: \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\") " pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:13:15.663040 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.663046 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-tls-cert\") pod \"authorino-588f75498-glhlq\" (UID: \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\") " pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:13:15.763673 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.763645 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q64qp\" (UniqueName: \"kubernetes.io/projected/bb383ff8-0436-4317-9ba7-26820ab05870-kube-api-access-q64qp\") pod \"bb383ff8-0436-4317-9ba7-26820ab05870\" (UID: \"bb383ff8-0436-4317-9ba7-26820ab05870\") " Aug 03 16:13:15.763854 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.763826 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v78rf\" (UniqueName: \"kubernetes.io/projected/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-kube-api-access-v78rf\") pod \"authorino-588f75498-glhlq\" (UID: \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\") " pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:13:15.763926 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.763854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-tls-cert\") pod \"authorino-588f75498-glhlq\" (UID: \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\") " pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:13:15.765717 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.765691 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb383ff8-0436-4317-9ba7-26820ab05870-kube-api-access-q64qp" (OuterVolumeSpecName: "kube-api-access-q64qp") pod "bb383ff8-0436-4317-9ba7-26820ab05870" (UID: "bb383ff8-0436-4317-9ba7-26820ab05870"). InnerVolumeSpecName "kube-api-access-q64qp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:13:15.766318 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.766301 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-tls-cert\") pod \"authorino-588f75498-glhlq\" (UID: \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\") " pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:13:15.773152 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.773126 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v78rf\" (UniqueName: \"kubernetes.io/projected/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-kube-api-access-v78rf\") pod \"authorino-588f75498-glhlq\" (UID: \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\") " pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:13:15.803860 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.803835 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:13:15.864609 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.864580 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q64qp\" (UniqueName: \"kubernetes.io/projected/bb383ff8-0436-4317-9ba7-26820ab05870-kube-api-access-q64qp\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:13:15.930761 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:15.930736 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-588f75498-glhlq"] Aug 03 16:13:15.931995 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:13:15.931972 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5a4dbdb_a845_4e5e_94ee_c0cc65b07993.slice/crio-35134e3ccf935884e42687d1a9c49c6294d231e3ca97e90c5b81ce6f8d5c1985 WatchSource:0}: Error finding container 35134e3ccf935884e42687d1a9c49c6294d231e3ca97e90c5b81ce6f8d5c1985: Status 404 returned error can't find the container with id 35134e3ccf935884e42687d1a9c49c6294d231e3ca97e90c5b81ce6f8d5c1985 Aug 03 16:13:16.644868 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:16.644841 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-xvfmk" Aug 03 16:13:16.644868 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:16.644849 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-588f75498-glhlq" event={"ID":"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993","Type":"ContainerStarted","Data":"35134e3ccf935884e42687d1a9c49c6294d231e3ca97e90c5b81ce6f8d5c1985"} Aug 03 16:13:16.672859 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:16.672831 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xvfmk"] Aug 03 16:13:16.676341 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:16.676318 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-xvfmk"] Aug 03 16:13:17.651324 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:17.651291 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-588f75498-glhlq" event={"ID":"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993","Type":"ContainerStarted","Data":"7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2"} Aug 03 16:13:17.668979 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:17.668927 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-588f75498-glhlq" podStartSLOduration=1.673803875 podStartE2EDuration="2.668913057s" totalImagesPullingTime="995.109182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:13:17.666574652 +0000 UTC m=+536.224657237" watchObservedRunningTime="2026-08-03 16:13:17.668913057 +0000 UTC m=+536.226995642" Aug 03 16:13:17.694947 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:17.694916 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d78rq"] Aug 03 16:13:17.695141 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:17.695117 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-d78rq" podUID="6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d" containerName="authorino" containerID="cri-o://1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060" gracePeriod=30 Aug 03 16:13:17.935665 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:17.935643 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-d78rq" Aug 03 16:13:17.984925 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:17.984893 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8h2c6\" (UniqueName: \"kubernetes.io/projected/6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d-kube-api-access-8h2c6\") pod \"6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d\" (UID: \"6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d\") " Aug 03 16:13:17.986856 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:17.986823 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d-kube-api-access-8h2c6" (OuterVolumeSpecName: "kube-api-access-8h2c6") pod "6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d" (UID: "6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d"). InnerVolumeSpecName "kube-api-access-8h2c6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:13:17.996470 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:17.996439 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb383ff8-0436-4317-9ba7-26820ab05870" path="/var/lib/kubelet/pods/bb383ff8-0436-4317-9ba7-26820ab05870/volumes" Aug 03 16:13:18.085480 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.085446 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8h2c6\" (UniqueName: \"kubernetes.io/projected/6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d-kube-api-access-8h2c6\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:13:18.657895 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.657858 2582 generic.go:356] "Generic (PLEG): container finished" podID="6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d" containerID="1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060" exitCode=0 Aug 03 16:13:18.658297 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.657903 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-d78rq" Aug 03 16:13:18.658297 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.657934 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-d78rq" event={"ID":"6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d","Type":"ContainerDied","Data":"1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060"} Aug 03 16:13:18.658297 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.657974 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-d78rq" event={"ID":"6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d","Type":"ContainerDied","Data":"6c539b190d41b5acfde27b68d3a492b10995efed97654e01cbeee93433cf211d"} Aug 03 16:13:18.658297 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.657997 2582 scope.go:117] "RemoveContainer" containerID="1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060" Aug 03 16:13:18.666216 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.666186 2582 scope.go:117] "RemoveContainer" containerID="1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060" Aug 03 16:13:18.666452 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:13:18.666437 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060\": container with ID starting with 1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060 not found: ID does not exist" containerID="1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060" Aug 03 16:13:18.666497 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.666459 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060"} err="failed to get container status \"1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060\": rpc error: code = NotFound desc = could not find container \"1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060\": container with ID starting with 1f56f07880bc64479efd8fcaa07cc2f826f664d5efc19d2d1dfddec82384f060 not found: ID does not exist" Aug 03 16:13:18.676247 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.676224 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d78rq"] Aug 03 16:13:18.686847 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:18.683168 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-d78rq"] Aug 03 16:13:19.996800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:13:19.996764 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d" path="/var/lib/kubelet/pods/6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d/volumes" Aug 03 16:14:04.902274 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:04.902178 2582 generic.go:356] "Generic (PLEG): container finished" podID="8db2abdc-9a5a-49a2-9df8-909dacd7cf61" containerID="8f3204fd519298b93de317a8bc1c852a5bf43ff97d576fca7b95db4b0a835363" exitCode=0 Aug 03 16:14:04.902274 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:04.902250 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" event={"ID":"8db2abdc-9a5a-49a2-9df8-909dacd7cf61","Type":"ContainerDied","Data":"8f3204fd519298b93de317a8bc1c852a5bf43ff97d576fca7b95db4b0a835363"} Aug 03 16:14:04.902735 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:04.902613 2582 scope.go:117] "RemoveContainer" containerID="8f3204fd519298b93de317a8bc1c852a5bf43ff97d576fca7b95db4b0a835363" Aug 03 16:14:04.903557 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:04.903533 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:14:04.903611 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:04.903599 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:14:04.903676 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:04.903645 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:14:05.909218 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:05.909186 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" event={"ID":"8db2abdc-9a5a-49a2-9df8-909dacd7cf61","Type":"ContainerStarted","Data":"f77d673c4033c4f76f1b51a6fea980f75d33e47f2c1eeddeb6c4197beb4aec92"} Aug 03 16:14:06.061795 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:06.061750 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:14:06.062989 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:06.062959 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-4htkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 03 16:14:06.062989 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:06.063010 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" podUID="8db2abdc-9a5a-49a2-9df8-909dacd7cf61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 03 16:14:06.412058 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:06.412025 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:14:07.062469 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:07.062425 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-4htkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 03 16:14:07.062908 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:07.062498 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" podUID="8db2abdc-9a5a-49a2-9df8-909dacd7cf61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 03 16:14:08.062203 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:08.062169 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-4htkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 03 16:14:08.062394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:08.062224 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" podUID="8db2abdc-9a5a-49a2-9df8-909dacd7cf61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 03 16:14:09.062069 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:09.062028 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-4htkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 03 16:14:09.062464 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:09.062097 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" podUID="8db2abdc-9a5a-49a2-9df8-909dacd7cf61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 03 16:14:10.062084 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:10.062041 2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-4htkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Aug 03 16:14:10.062467 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:10.062104 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" podUID="8db2abdc-9a5a-49a2-9df8-909dacd7cf61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Aug 03 16:14:11.134171 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:11.134138 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:14:11.137176 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:11.137149 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4htkn" Aug 03 16:14:19.549106 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.549069 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2"] Aug 03 16:14:19.549554 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.549535 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d" containerName="authorino" Aug 03 16:14:19.549600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.549559 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d" containerName="authorino" Aug 03 16:14:19.549703 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.549689 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e0601f5-f5c3-44e3-bfd1-77e1956e5c8d" containerName="authorino" Aug 03 16:14:19.551805 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.551788 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.555909 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.555887 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 03 16:14:19.555909 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.555905 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Aug 03 16:14:19.556095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.555897 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6svrh\"" Aug 03 16:14:19.556095 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.556001 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 03 16:14:19.565563 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.565541 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2"] Aug 03 16:14:19.626148 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.626114 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6nbx\" (UniqueName: \"kubernetes.io/projected/103c8f44-468a-469b-a846-127395769464-kube-api-access-m6nbx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.626148 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.626151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/103c8f44-468a-469b-a846-127395769464-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.626348 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.626171 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.626348 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.626241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.626348 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.626260 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.626348 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.626289 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.726686 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.726649 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.726914 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.726723 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.726914 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.726750 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.726914 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.726782 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.727072 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.726916 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6nbx\" (UniqueName: \"kubernetes.io/projected/103c8f44-468a-469b-a846-127395769464-kube-api-access-m6nbx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.727072 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.726971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/103c8f44-468a-469b-a846-127395769464-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.727148 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.727082 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.727184 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.727146 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.727184 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.727174 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.729046 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.729024 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/103c8f44-468a-469b-a846-127395769464-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.729235 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.729219 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/103c8f44-468a-469b-a846-127395769464-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.735925 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.735901 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6nbx\" (UniqueName: \"kubernetes.io/projected/103c8f44-468a-469b-a846-127395769464-kube-api-access-m6nbx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j2sr2\" (UID: \"103c8f44-468a-469b-a846-127395769464\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.862455 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.862377 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:19.993538 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:14:19.993509 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103c8f44_468a_469b_a846_127395769464.slice/crio-be0baf5e3b3d7e64fadb56d7bfa006dd45bd9528f552167b56583e7e5b97a747 WatchSource:0}: Error finding container be0baf5e3b3d7e64fadb56d7bfa006dd45bd9528f552167b56583e7e5b97a747: Status 404 returned error can't find the container with id be0baf5e3b3d7e64fadb56d7bfa006dd45bd9528f552167b56583e7e5b97a747 Aug 03 16:14:19.997744 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:19.997644 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2"] Aug 03 16:14:20.994540 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:20.994512 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" event={"ID":"103c8f44-468a-469b-a846-127395769464","Type":"ContainerStarted","Data":"be0baf5e3b3d7e64fadb56d7bfa006dd45bd9528f552167b56583e7e5b97a747"} Aug 03 16:14:22.000862 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:22.000832 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" event={"ID":"103c8f44-468a-469b-a846-127395769464","Type":"ContainerStarted","Data":"e5f06f9bdd65502abd26c937f032e98dd30eff1d1ed29e259727782aed3fe93b"} Aug 03 16:14:22.020020 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:22.019975 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" podStartSLOduration=1.9601969590000001 podStartE2EDuration="3.019959197s" totalImagesPullingTime="1.059762238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:14:22.018026313 +0000 UTC m=+600.576108898" watchObservedRunningTime="2026-08-03 16:14:22.019959197 +0000 UTC m=+600.578041781" Aug 03 16:14:29.863148 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:29.863107 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:29.863700 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:29.863180 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:29.875293 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:29.875270 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:30.052726 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:30.052693 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j2sr2" Aug 03 16:14:51.848791 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.848711 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z"] Aug 03 16:14:51.851280 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.851265 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:51.854251 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.854225 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Aug 03 16:14:51.864272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.864250 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z"] Aug 03 16:14:51.914846 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.914811 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:51.914846 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.914853 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:51.915081 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.914905 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2q4g\" (UniqueName: \"kubernetes.io/projected/47d84110-6dcc-421c-b1df-a1c79c825ebe-kube-api-access-r2q4g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:51.915081 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.914927 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47d84110-6dcc-421c-b1df-a1c79c825ebe-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:51.915081 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.914970 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:51.915081 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:51.915008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.015966 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.015936 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.016138 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.015991 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2q4g\" (UniqueName: \"kubernetes.io/projected/47d84110-6dcc-421c-b1df-a1c79c825ebe-kube-api-access-r2q4g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.016138 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.016012 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47d84110-6dcc-421c-b1df-a1c79c825ebe-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.016138 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.016049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.016138 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.016074 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.016138 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.016097 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.016449 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.016427 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.016520 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.016456 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.016579 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.016556 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.018109 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.018092 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47d84110-6dcc-421c-b1df-a1c79c825ebe-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.018451 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.018435 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47d84110-6dcc-421c-b1df-a1c79c825ebe-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.026010 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.025989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2q4g\" (UniqueName: \"kubernetes.io/projected/47d84110-6dcc-421c-b1df-a1c79c825ebe-kube-api-access-r2q4g\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z\" (UID: \"47d84110-6dcc-421c-b1df-a1c79c825ebe\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.164217 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.164123 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:14:52.294897 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.294871 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z"] Aug 03 16:14:52.296560 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:14:52.296535 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d84110_6dcc_421c_b1df_a1c79c825ebe.slice/crio-b82f7391c48debd428d7c9fd119e3a73e5863f34c578da7224d033fdaea46248 WatchSource:0}: Error finding container b82f7391c48debd428d7c9fd119e3a73e5863f34c578da7224d033fdaea46248: Status 404 returned error can't find the container with id b82f7391c48debd428d7c9fd119e3a73e5863f34c578da7224d033fdaea46248 Aug 03 16:14:52.297776 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:52.297758 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 03 16:14:53.162031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:53.161993 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" event={"ID":"47d84110-6dcc-421c-b1df-a1c79c825ebe","Type":"ContainerStarted","Data":"741a3aa9fae9c458cc94e29e34f8fcd69b78c0009f5ac258e2543bb5c69b8082"} Aug 03 16:14:53.162532 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:53.162054 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" event={"ID":"47d84110-6dcc-421c-b1df-a1c79c825ebe","Type":"ContainerStarted","Data":"b82f7391c48debd428d7c9fd119e3a73e5863f34c578da7224d033fdaea46248"} Aug 03 16:14:53.182398 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:14:53.182346 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" podStartSLOduration=1.8122596020000001 podStartE2EDuration="2.18232975s" totalImagesPullingTime="370.070148ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:14:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:14:53.180724693 +0000 UTC m=+631.738807281" watchObservedRunningTime="2026-08-03 16:14:53.18232975 +0000 UTC m=+631.740412334" Aug 03 16:15:02.164716 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:02.164674 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:15:02.165299 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:02.164738 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:15:02.178936 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:02.178906 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:15:02.227580 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:02.227543 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z" Aug 03 16:15:56.269878 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.269847 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dc447499-lwc8r"] Aug 03 16:15:56.272288 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.272272 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dc447499-lwc8r" Aug 03 16:15:56.281807 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.281785 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dc447499-lwc8r"] Aug 03 16:15:56.367882 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.367849 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07341eab-9128-49cf-948a-b14235ad42da-tls-cert\") pod \"authorino-6dc447499-lwc8r\" (UID: \"07341eab-9128-49cf-948a-b14235ad42da\") " pod="kuadrant-system/authorino-6dc447499-lwc8r" Aug 03 16:15:56.368099 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.367889 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h22wx\" (UniqueName: \"kubernetes.io/projected/07341eab-9128-49cf-948a-b14235ad42da-kube-api-access-h22wx\") pod \"authorino-6dc447499-lwc8r\" (UID: \"07341eab-9128-49cf-948a-b14235ad42da\") " pod="kuadrant-system/authorino-6dc447499-lwc8r" Aug 03 16:15:56.469235 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.469197 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07341eab-9128-49cf-948a-b14235ad42da-tls-cert\") pod \"authorino-6dc447499-lwc8r\" (UID: \"07341eab-9128-49cf-948a-b14235ad42da\") " pod="kuadrant-system/authorino-6dc447499-lwc8r" Aug 03 16:15:56.469235 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.469230 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h22wx\" (UniqueName: \"kubernetes.io/projected/07341eab-9128-49cf-948a-b14235ad42da-kube-api-access-h22wx\") pod \"authorino-6dc447499-lwc8r\" (UID: \"07341eab-9128-49cf-948a-b14235ad42da\") " pod="kuadrant-system/authorino-6dc447499-lwc8r" Aug 03 16:15:56.471567 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.471546 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/07341eab-9128-49cf-948a-b14235ad42da-tls-cert\") pod \"authorino-6dc447499-lwc8r\" (UID: \"07341eab-9128-49cf-948a-b14235ad42da\") " pod="kuadrant-system/authorino-6dc447499-lwc8r" Aug 03 16:15:56.479348 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.479322 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h22wx\" (UniqueName: \"kubernetes.io/projected/07341eab-9128-49cf-948a-b14235ad42da-kube-api-access-h22wx\") pod \"authorino-6dc447499-lwc8r\" (UID: \"07341eab-9128-49cf-948a-b14235ad42da\") " pod="kuadrant-system/authorino-6dc447499-lwc8r" Aug 03 16:15:56.581750 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.581672 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dc447499-lwc8r" Aug 03 16:15:56.716315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:56.716286 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dc447499-lwc8r"] Aug 03 16:15:56.718253 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:15:56.718228 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07341eab_9128_49cf_948a_b14235ad42da.slice/crio-52a6c9b0e95b795b1dbd8bb68b8e130c572e6a6be73b1f0314cb304fd87f0cd0 WatchSource:0}: Error finding container 52a6c9b0e95b795b1dbd8bb68b8e130c572e6a6be73b1f0314cb304fd87f0cd0: Status 404 returned error can't find the container with id 52a6c9b0e95b795b1dbd8bb68b8e130c572e6a6be73b1f0314cb304fd87f0cd0 Aug 03 16:15:57.516593 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:57.516553 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dc447499-lwc8r" event={"ID":"07341eab-9128-49cf-948a-b14235ad42da","Type":"ContainerStarted","Data":"52a6c9b0e95b795b1dbd8bb68b8e130c572e6a6be73b1f0314cb304fd87f0cd0"} Aug 03 16:15:58.525540 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.525506 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dc447499-lwc8r" event={"ID":"07341eab-9128-49cf-948a-b14235ad42da","Type":"ContainerStarted","Data":"23e0975c4a3cee0b9b2d04f15681fb1de87a3014729dceb839364a74173005fb"} Aug 03 16:15:58.544381 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.544324 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6dc447499-lwc8r" podStartSLOduration=1.666825963 podStartE2EDuration="2.544307221s" totalImagesPullingTime="877.481258ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:15:58.543547762 +0000 UTC m=+697.101630370" watchObservedRunningTime="2026-08-03 16:15:58.544307221 +0000 UTC m=+697.102389805" Aug 03 16:15:58.570829 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.570800 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-588f75498-glhlq"] Aug 03 16:15:58.571120 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.571091 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-588f75498-glhlq" podUID="b5a4dbdb-a845-4e5e-94ee-c0cc65b07993" containerName="authorino" containerID="cri-o://7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2" gracePeriod=30 Aug 03 16:15:58.819530 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.819499 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:15:58.894456 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.894424 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v78rf\" (UniqueName: \"kubernetes.io/projected/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-kube-api-access-v78rf\") pod \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\" (UID: \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\") " Aug 03 16:15:58.894657 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.894486 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-tls-cert\") pod \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\" (UID: \"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993\") " Aug 03 16:15:58.896454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.896423 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-kube-api-access-v78rf" (OuterVolumeSpecName: "kube-api-access-v78rf") pod "b5a4dbdb-a845-4e5e-94ee-c0cc65b07993" (UID: "b5a4dbdb-a845-4e5e-94ee-c0cc65b07993"). InnerVolumeSpecName "kube-api-access-v78rf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:15:58.906109 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.906080 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b5a4dbdb-a845-4e5e-94ee-c0cc65b07993" (UID: "b5a4dbdb-a845-4e5e-94ee-c0cc65b07993"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:15:58.995486 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.995453 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v78rf\" (UniqueName: \"kubernetes.io/projected/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-kube-api-access-v78rf\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:15:58.995486 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:58.995479 2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993-tls-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:15:59.532081 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.532044 2582 generic.go:356] "Generic (PLEG): container finished" podID="b5a4dbdb-a845-4e5e-94ee-c0cc65b07993" containerID="7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2" exitCode=0 Aug 03 16:15:59.532081 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.532086 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-588f75498-glhlq" Aug 03 16:15:59.532650 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.532134 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-588f75498-glhlq" event={"ID":"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993","Type":"ContainerDied","Data":"7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2"} Aug 03 16:15:59.532650 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.532179 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-588f75498-glhlq" event={"ID":"b5a4dbdb-a845-4e5e-94ee-c0cc65b07993","Type":"ContainerDied","Data":"35134e3ccf935884e42687d1a9c49c6294d231e3ca97e90c5b81ce6f8d5c1985"} Aug 03 16:15:59.532650 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.532203 2582 scope.go:117] "RemoveContainer" containerID="7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2" Aug 03 16:15:59.542653 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.542613 2582 scope.go:117] "RemoveContainer" containerID="7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2" Aug 03 16:15:59.542942 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:15:59.542923 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2\": container with ID starting with 7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2 not found: ID does not exist" containerID="7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2" Aug 03 16:15:59.543025 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.542957 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2"} err="failed to get container status \"7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2\": rpc error: code = NotFound desc = could not find container \"7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2\": container with ID starting with 7fed4ba440c9fe86e757420a3eca4d679f183578be0676e9fd5c9d7a0c635bd2 not found: ID does not exist" Aug 03 16:15:59.566728 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.566699 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-588f75498-glhlq"] Aug 03 16:15:59.571165 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.571144 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-588f75498-glhlq"] Aug 03 16:15:59.995805 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:15:59.995774 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5a4dbdb-a845-4e5e-94ee-c0cc65b07993" path="/var/lib/kubelet/pods/b5a4dbdb-a845-4e5e-94ee-c0cc65b07993/volumes" Aug 03 16:24:34.093078 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.093046 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s"] Aug 03 16:24:34.095644 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.093461 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" podUID="7298fbb9-d217-4c0a-9289-1dbeab889ded" containerName="manager" containerID="cri-o://4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae" gracePeriod=10 Aug 03 16:24:34.334828 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.334806 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:24:34.374464 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.374378 2582 generic.go:356] "Generic (PLEG): container finished" podID="7298fbb9-d217-4c0a-9289-1dbeab889ded" containerID="4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae" exitCode=0 Aug 03 16:24:34.374464 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.374413 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" Aug 03 16:24:34.374464 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.374428 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" event={"ID":"7298fbb9-d217-4c0a-9289-1dbeab889ded","Type":"ContainerDied","Data":"4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae"} Aug 03 16:24:34.374741 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.374473 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s" event={"ID":"7298fbb9-d217-4c0a-9289-1dbeab889ded","Type":"ContainerDied","Data":"71a048850307cc8f472130db9e4a50bb0f55f1bdf00153d817145c77fc19bf41"} Aug 03 16:24:34.374741 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.374500 2582 scope.go:117] "RemoveContainer" containerID="4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae" Aug 03 16:24:34.385704 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.385687 2582 scope.go:117] "RemoveContainer" containerID="4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae" Aug 03 16:24:34.385997 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:24:34.385968 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae\": container with ID starting with 4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae not found: ID does not exist" containerID="4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae" Aug 03 16:24:34.386089 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.386005 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae"} err="failed to get container status \"4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae\": rpc error: code = NotFound desc = could not find container \"4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae\": container with ID starting with 4c453b2511e1452da74956b95199912223eb1dbbb7c88b73575d62d5826660ae not found: ID does not exist" Aug 03 16:24:34.395967 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.395942 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56hrg\" (UniqueName: \"kubernetes.io/projected/7298fbb9-d217-4c0a-9289-1dbeab889ded-kube-api-access-56hrg\") pod \"7298fbb9-d217-4c0a-9289-1dbeab889ded\" (UID: \"7298fbb9-d217-4c0a-9289-1dbeab889ded\") " Aug 03 16:24:34.396089 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.396009 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7298fbb9-d217-4c0a-9289-1dbeab889ded-extensions-socket-volume\") pod \"7298fbb9-d217-4c0a-9289-1dbeab889ded\" (UID: \"7298fbb9-d217-4c0a-9289-1dbeab889ded\") " Aug 03 16:24:34.396377 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.396356 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7298fbb9-d217-4c0a-9289-1dbeab889ded-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7298fbb9-d217-4c0a-9289-1dbeab889ded" (UID: "7298fbb9-d217-4c0a-9289-1dbeab889ded"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:24:34.398249 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.398225 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7298fbb9-d217-4c0a-9289-1dbeab889ded-kube-api-access-56hrg" (OuterVolumeSpecName: "kube-api-access-56hrg") pod "7298fbb9-d217-4c0a-9289-1dbeab889ded" (UID: "7298fbb9-d217-4c0a-9289-1dbeab889ded"). InnerVolumeSpecName "kube-api-access-56hrg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:24:34.496627 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.496584 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56hrg\" (UniqueName: \"kubernetes.io/projected/7298fbb9-d217-4c0a-9289-1dbeab889ded-kube-api-access-56hrg\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:24:34.496627 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.496612 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7298fbb9-d217-4c0a-9289-1dbeab889ded-extensions-socket-volume\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:24:34.719893 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.719862 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s"] Aug 03 16:24:34.726607 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:34.726582 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-t5d4s"] Aug 03 16:24:35.994426 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:24:35.994396 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7298fbb9-d217-4c0a-9289-1dbeab889ded" path="/var/lib/kubelet/pods/7298fbb9-d217-4c0a-9289-1dbeab889ded/volumes" Aug 03 16:25:34.798880 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.798848 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8"] Aug 03 16:25:34.799278 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.799247 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7298fbb9-d217-4c0a-9289-1dbeab889ded" containerName="manager" Aug 03 16:25:34.799278 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.799261 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7298fbb9-d217-4c0a-9289-1dbeab889ded" containerName="manager" Aug 03 16:25:34.799278 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.799269 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a4dbdb-a845-4e5e-94ee-c0cc65b07993" containerName="authorino" Aug 03 16:25:34.799278 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.799275 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a4dbdb-a845-4e5e-94ee-c0cc65b07993" containerName="authorino" Aug 03 16:25:34.799404 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.799340 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7298fbb9-d217-4c0a-9289-1dbeab889ded" containerName="manager" Aug 03 16:25:34.799404 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.799352 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5a4dbdb-a845-4e5e-94ee-c0cc65b07993" containerName="authorino" Aug 03 16:25:34.803421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.803397 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:25:34.807813 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.807786 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lwnjp\"" Aug 03 16:25:34.820116 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.820090 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8"] Aug 03 16:25:34.936352 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.936321 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44qmd\" (UniqueName: \"kubernetes.io/projected/f24fab85-6f74-42d9-a16f-e154055e18ca-kube-api-access-44qmd\") pod \"kuadrant-operator-controller-manager-68cf589589-xd6f8\" (UID: \"f24fab85-6f74-42d9-a16f-e154055e18ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:25:34.936534 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:34.936360 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24fab85-6f74-42d9-a16f-e154055e18ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-xd6f8\" (UID: \"f24fab85-6f74-42d9-a16f-e154055e18ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:25:35.037375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.037335 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44qmd\" (UniqueName: \"kubernetes.io/projected/f24fab85-6f74-42d9-a16f-e154055e18ca-kube-api-access-44qmd\") pod \"kuadrant-operator-controller-manager-68cf589589-xd6f8\" (UID: \"f24fab85-6f74-42d9-a16f-e154055e18ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:25:35.037375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.037382 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24fab85-6f74-42d9-a16f-e154055e18ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-xd6f8\" (UID: \"f24fab85-6f74-42d9-a16f-e154055e18ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:25:35.037787 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.037767 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24fab85-6f74-42d9-a16f-e154055e18ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-xd6f8\" (UID: \"f24fab85-6f74-42d9-a16f-e154055e18ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:25:35.046977 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.046957 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44qmd\" (UniqueName: \"kubernetes.io/projected/f24fab85-6f74-42d9-a16f-e154055e18ca-kube-api-access-44qmd\") pod \"kuadrant-operator-controller-manager-68cf589589-xd6f8\" (UID: \"f24fab85-6f74-42d9-a16f-e154055e18ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:25:35.112761 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.112685 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:25:35.255528 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.255504 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8"] Aug 03 16:25:35.258127 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:25:35.258100 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf24fab85_6f74_42d9_a16f_e154055e18ca.slice/crio-6f8394d163f62a33b14a64c7e1ead3f56b880abaac15bf35046cc311c4101030 WatchSource:0}: Error finding container 6f8394d163f62a33b14a64c7e1ead3f56b880abaac15bf35046cc311c4101030: Status 404 returned error can't find the container with id 6f8394d163f62a33b14a64c7e1ead3f56b880abaac15bf35046cc311c4101030 Aug 03 16:25:35.694809 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.694779 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" event={"ID":"f24fab85-6f74-42d9-a16f-e154055e18ca","Type":"ContainerStarted","Data":"592de2ef058bb8638dbe57bff3e9d555aed3d31533c30b92e58f972ca8857af2"} Aug 03 16:25:35.694980 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.694830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" event={"ID":"f24fab85-6f74-42d9-a16f-e154055e18ca","Type":"ContainerStarted","Data":"6f8394d163f62a33b14a64c7e1ead3f56b880abaac15bf35046cc311c4101030"} Aug 03 16:25:35.694980 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.694861 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:25:35.728164 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:35.728100 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" podStartSLOduration=1.728079838 podStartE2EDuration="1.728079838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:25:35.720808909 +0000 UTC m=+1274.278891547" watchObservedRunningTime="2026-08-03 16:25:35.728079838 +0000 UTC m=+1274.286162423" Aug 03 16:25:46.703492 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:25:46.703456 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:28:45.452550 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.452515 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q"] Aug 03 16:28:45.456040 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.456020 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.459104 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.459079 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ed4bc549-kserve-self-signed-certs\"" Aug 03 16:28:45.465180 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.465161 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q"] Aug 03 16:28:45.527116 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.527090 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-model-cache\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.527286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.527126 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6wd8\" (UniqueName: \"kubernetes.io/projected/0521ebb8-2f00-4125-bd6c-48c02cc5520e-kube-api-access-x6wd8\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.527286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.527145 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-home\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.527286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.527162 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tls-certs\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.527286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.527234 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-dshm\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.527286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.527259 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tmp-dir\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.628446 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.628409 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6wd8\" (UniqueName: \"kubernetes.io/projected/0521ebb8-2f00-4125-bd6c-48c02cc5520e-kube-api-access-x6wd8\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.628446 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.628447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-home\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.628682 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.628468 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tls-certs\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.628682 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.628662 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-dshm\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.628782 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.628702 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tmp-dir\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.628852 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.628803 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-model-cache\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.628852 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.628830 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-home\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.629066 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.629040 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tmp-dir\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.629066 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.629067 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-model-cache\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.630763 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.630740 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-dshm\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.630900 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.630885 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tls-certs\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.637897 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.637864 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6wd8\" (UniqueName: \"kubernetes.io/projected/0521ebb8-2f00-4125-bd6c-48c02cc5520e-kube-api-access-x6wd8\") pod \"e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.766235 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.766208 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:45.900349 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:45.900321 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q"] Aug 03 16:28:45.901043 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:28:45.901018 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0521ebb8_2f00_4125_bd6c_48c02cc5520e.slice/crio-6ceb5e5fe36f3cd5c4a1c0f9219e3aa7ee933b29468b29df36fdc42b690a6dd2 WatchSource:0}: Error finding container 6ceb5e5fe36f3cd5c4a1c0f9219e3aa7ee933b29468b29df36fdc42b690a6dd2: Status 404 returned error can't find the container with id 6ceb5e5fe36f3cd5c4a1c0f9219e3aa7ee933b29468b29df36fdc42b690a6dd2 Aug 03 16:28:46.750436 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:46.750399 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" event={"ID":"0521ebb8-2f00-4125-bd6c-48c02cc5520e","Type":"ContainerStarted","Data":"fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54"} Aug 03 16:28:46.750914 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:46.750447 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" event={"ID":"0521ebb8-2f00-4125-bd6c-48c02cc5520e","Type":"ContainerStarted","Data":"6ceb5e5fe36f3cd5c4a1c0f9219e3aa7ee933b29468b29df36fdc42b690a6dd2"} Aug 03 16:28:46.770073 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:46.770025 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" podStartSLOduration=1.770009387 podStartE2EDuration="1.770009387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:28:46.766951351 +0000 UTC m=+1465.325033935" watchObservedRunningTime="2026-08-03 16:28:46.770009387 +0000 UTC m=+1465.328091972" Aug 03 16:28:55.767206 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:55.767173 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:55.767711 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:55.767219 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:55.779168 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:55.779147 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:28:55.810564 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:28:55.810539 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:29:12.681775 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:12.681745 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q"] Aug 03 16:29:27.700087 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:27.700026 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" podUID="0521ebb8-2f00-4125-bd6c-48c02cc5520e" containerName="main" containerID="cri-o://fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54" gracePeriod=45 Aug 03 16:29:27.941996 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:27.941966 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:29:27.979863 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:27.979774 2582 generic.go:356] "Generic (PLEG): container finished" podID="0521ebb8-2f00-4125-bd6c-48c02cc5520e" containerID="fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54" exitCode=0 Aug 03 16:29:27.979863 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:27.979816 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" Aug 03 16:29:27.980096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:27.979861 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" event={"ID":"0521ebb8-2f00-4125-bd6c-48c02cc5520e","Type":"ContainerDied","Data":"fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54"} Aug 03 16:29:27.980096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:27.979899 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q" event={"ID":"0521ebb8-2f00-4125-bd6c-48c02cc5520e","Type":"ContainerDied","Data":"6ceb5e5fe36f3cd5c4a1c0f9219e3aa7ee933b29468b29df36fdc42b690a6dd2"} Aug 03 16:29:27.980096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:27.979918 2582 scope.go:117] "RemoveContainer" containerID="fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54" Aug 03 16:29:27.988352 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:27.988333 2582 scope.go:117] "RemoveContainer" containerID="fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54" Aug 03 16:29:27.988636 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:29:27.988599 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54\": container with ID starting with fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54 not found: ID does not exist" containerID="fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54" Aug 03 16:29:27.988730 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:27.988645 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54"} err="failed to get container status \"fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54\": rpc error: code = NotFound desc = could not find container \"fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54\": container with ID starting with fb3cccad85b87923df8dee90823a127367054950074b8d90b36f8ff4a08c4f54 not found: ID does not exist" Aug 03 16:29:28.005820 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.005801 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-home\") pod \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " Aug 03 16:29:28.005923 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.005842 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-dshm\") pod \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " Aug 03 16:29:28.006033 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.006013 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-home" (OuterVolumeSpecName: "home") pod "0521ebb8-2f00-4125-bd6c-48c02cc5520e" (UID: "0521ebb8-2f00-4125-bd6c-48c02cc5520e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:29:28.007822 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.007803 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-dshm" (OuterVolumeSpecName: "dshm") pod "0521ebb8-2f00-4125-bd6c-48c02cc5520e" (UID: "0521ebb8-2f00-4125-bd6c-48c02cc5520e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:29:28.106370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.106336 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6wd8\" (UniqueName: \"kubernetes.io/projected/0521ebb8-2f00-4125-bd6c-48c02cc5520e-kube-api-access-x6wd8\") pod \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " Aug 03 16:29:28.106370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.106382 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tmp-dir\") pod \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " Aug 03 16:29:28.106605 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.106526 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tls-certs\") pod \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " Aug 03 16:29:28.106605 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.106554 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-model-cache\") pod \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\" (UID: \"0521ebb8-2f00-4125-bd6c-48c02cc5520e\") " Aug 03 16:29:28.106754 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.106713 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0521ebb8-2f00-4125-bd6c-48c02cc5520e" (UID: "0521ebb8-2f00-4125-bd6c-48c02cc5520e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:29:28.106844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.106823 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-model-cache" (OuterVolumeSpecName: "model-cache") pod "0521ebb8-2f00-4125-bd6c-48c02cc5520e" (UID: "0521ebb8-2f00-4125-bd6c-48c02cc5520e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:29:28.106912 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.106835 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tmp-dir\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:29:28.106912 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.106863 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-home\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:29:28.106912 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.106872 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-dshm\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:29:28.108469 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.108449 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0521ebb8-2f00-4125-bd6c-48c02cc5520e" (UID: "0521ebb8-2f00-4125-bd6c-48c02cc5520e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:29:28.108538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.108514 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0521ebb8-2f00-4125-bd6c-48c02cc5520e-kube-api-access-x6wd8" (OuterVolumeSpecName: "kube-api-access-x6wd8") pod "0521ebb8-2f00-4125-bd6c-48c02cc5520e" (UID: "0521ebb8-2f00-4125-bd6c-48c02cc5520e"). InnerVolumeSpecName "kube-api-access-x6wd8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:29:28.207839 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.207811 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0521ebb8-2f00-4125-bd6c-48c02cc5520e-tls-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:29:28.207839 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.207833 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0521ebb8-2f00-4125-bd6c-48c02cc5520e-model-cache\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:29:28.207839 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.207841 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6wd8\" (UniqueName: \"kubernetes.io/projected/0521ebb8-2f00-4125-bd6c-48c02cc5520e-kube-api-access-x6wd8\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:29:28.303974 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.303943 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q"] Aug 03 16:29:28.308303 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:28.308281 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ed4bc549-kserve-766cbfc47-wm27q"] Aug 03 16:29:29.995800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:29:29.995766 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0521ebb8-2f00-4125-bd6c-48c02cc5520e" path="/var/lib/kubelet/pods/0521ebb8-2f00-4125-bd6c-48c02cc5520e/volumes" Aug 03 16:33:12.906704 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.906669 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf"] Aug 03 16:33:12.907279 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.907060 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0521ebb8-2f00-4125-bd6c-48c02cc5520e" containerName="main" Aug 03 16:33:12.907279 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.907072 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0521ebb8-2f00-4125-bd6c-48c02cc5520e" containerName="main" Aug 03 16:33:12.907279 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.907144 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0521ebb8-2f00-4125-bd6c-48c02cc5520e" containerName="main" Aug 03 16:33:12.910171 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.910153 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:12.913294 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.913276 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-77570ccf-openshift-default-dockercfg-227r4\"" Aug 03 16:33:12.931909 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.931887 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf"] Aug 03 16:33:12.994014 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.993980 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2390e10-2101-4332-bee1-b4f17fd35c7a-istiod-ca-cert\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:12.994014 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.994016 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-envoy\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:12.994222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.994037 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-socket\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:12.994222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.994067 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-certs\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:12.994222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.994092 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-token\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:12.994222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.994131 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-credential-socket\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:12.994222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.994157 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-podinfo\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:12.994222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.994178 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbx4s\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-kube-api-access-gbx4s\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:12.994222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:12.994200 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-data\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.094861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.094825 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-certs\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.094861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.094858 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-token\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095101 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.094884 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-credential-socket\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095101 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.094911 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-podinfo\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095101 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.095088 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbx4s\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-kube-api-access-gbx4s\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095262 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.095122 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-data\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095262 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.095158 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2390e10-2101-4332-bee1-b4f17fd35c7a-istiod-ca-cert\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095262 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.095189 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-envoy\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095262 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.095238 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-socket\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095478 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.095270 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-certs\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095716 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.095694 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-credential-socket\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095796 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.095743 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-data\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.095837 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.095809 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-socket\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.096040 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.096016 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2390e10-2101-4332-bee1-b4f17fd35c7a-istiod-ca-cert\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.097387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.097367 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-envoy\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.097633 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.097594 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-podinfo\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.103762 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.103734 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-token\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.103906 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.103888 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbx4s\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-kube-api-access-gbx4s\") pod \"e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.220407 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.220320 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:13.359947 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.359923 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf"] Aug 03 16:33:13.361470 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:33:13.361444 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2390e10_2101_4332_bee1_b4f17fd35c7a.slice/crio-1e880cd166e4c27e47531d0a28ba3278f73fa3d09fdfc288fbf7b11ce5cddae0 WatchSource:0}: Error finding container 1e880cd166e4c27e47531d0a28ba3278f73fa3d09fdfc288fbf7b11ce5cddae0: Status 404 returned error can't find the container with id 1e880cd166e4c27e47531d0a28ba3278f73fa3d09fdfc288fbf7b11ce5cddae0 Aug 03 16:33:13.363415 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.363380 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:33:13.363491 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.363443 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:33:13.363491 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:13.363476 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:33:14.238766 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:14.238730 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" event={"ID":"b2390e10-2101-4332-bee1-b4f17fd35c7a","Type":"ContainerStarted","Data":"7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504"} Aug 03 16:33:14.239160 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:14.238780 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" event={"ID":"b2390e10-2101-4332-bee1-b4f17fd35c7a","Type":"ContainerStarted","Data":"1e880cd166e4c27e47531d0a28ba3278f73fa3d09fdfc288fbf7b11ce5cddae0"} Aug 03 16:33:14.265587 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:14.265538 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" podStartSLOduration=2.265523315 podStartE2EDuration="2.265523315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:33:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:33:14.263771273 +0000 UTC m=+1732.821853858" watchObservedRunningTime="2026-08-03 16:33:14.265523315 +0000 UTC m=+1732.823605900" Aug 03 16:33:15.220888 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:15.220853 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:15.226008 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:15.225983 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:15.245784 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:15.245761 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:15.246610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:15.246591 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:28.318770 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:28.318732 2582 generic.go:356] "Generic (PLEG): container finished" podID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerID="7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504" exitCode=0 Aug 03 16:33:28.319200 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:28.318806 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" event={"ID":"b2390e10-2101-4332-bee1-b4f17fd35c7a","Type":"ContainerDied","Data":"7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504"} Aug 03 16:33:28.319200 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:28.319177 2582 scope.go:117] "RemoveContainer" containerID="7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504" Aug 03 16:33:28.320168 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:28.320140 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:33:28.320223 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:28.320209 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:33:28.320268 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:28.320239 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:33:29.325276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:29.325239 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" event={"ID":"b2390e10-2101-4332-bee1-b4f17fd35c7a","Type":"ContainerStarted","Data":"41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca"} Aug 03 16:33:30.220703 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:30.220667 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:30.221836 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:30.221813 2582 patch_prober.go:28] interesting pod/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 03 16:33:30.221947 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:30.221873 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 03 16:33:30.246512 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:30.246448 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:31.221524 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:31.221491 2582 patch_prober.go:28] interesting pod/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 03 16:33:31.221937 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:31.221541 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 03 16:33:32.221637 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:32.221586 2582 patch_prober.go:28] interesting pod/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 03 16:33:32.222048 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:32.221673 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 03 16:33:33.221463 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:33.221427 2582 patch_prober.go:28] interesting pod/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Aug 03 16:33:33.221678 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:33.221485 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Aug 03 16:33:34.225278 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:34.225252 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:34.226272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:34.226254 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:47.564836 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:47.564802 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf"] Aug 03 16:33:47.565481 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:47.565008 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" containerID="cri-o://41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca" gracePeriod=30 Aug 03 16:33:48.664737 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.664700 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g"] Aug 03 16:33:48.667450 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.667435 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.672582 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.672563 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-51e95abc-gw-openshift-default-dockercfg-tndz4\"" Aug 03 16:33:48.694007 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.693983 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g"] Aug 03 16:33:48.730078 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.730038 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w7fg\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-kube-api-access-9w7fg\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.730284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.730086 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-socket\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.730284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.730115 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istiod-ca-cert\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.730284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.730163 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-envoy\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.730284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.730198 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-data\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.730284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.730234 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-credential-socket\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.730284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.730277 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-certs\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.730578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.730315 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-token\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.730578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.730438 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-podinfo\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831149 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831117 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-envoy\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831329 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831158 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-data\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831329 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831193 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-credential-socket\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831329 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831215 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-certs\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831329 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831234 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-token\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831329 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831307 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-podinfo\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831595 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831353 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w7fg\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-kube-api-access-9w7fg\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831595 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-socket\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831595 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831401 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istiod-ca-cert\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831812 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831648 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-credential-socket\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.831928 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.831906 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-data\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.832169 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.832095 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istiod-ca-cert\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.832250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.832238 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-socket\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.832322 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.832303 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-certs\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.833453 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.833434 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-envoy\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.834106 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.834083 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-podinfo\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.842329 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.842310 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-token\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.847210 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.847193 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w7fg\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-kube-api-access-9w7fg\") pod \"e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:48.977505 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:48.977418 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:49.118494 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.118462 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g"] Aug 03 16:33:49.120186 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:33:49.120163 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9a1f99_6cd1_4665_819a_5f61ceb50998.slice/crio-8ab4838c3ffe63757e9d996d5776bbe159b4dd8021df8bd4ba152caf765a85b8 WatchSource:0}: Error finding container 8ab4838c3ffe63757e9d996d5776bbe159b4dd8021df8bd4ba152caf765a85b8: Status 404 returned error can't find the container with id 8ab4838c3ffe63757e9d996d5776bbe159b4dd8021df8bd4ba152caf765a85b8 Aug 03 16:33:49.122191 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.122160 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:33:49.122253 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.122224 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:33:49.122292 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.122253 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:33:49.227899 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.227815 2582 patch_prober.go:28] interesting pod/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 03 16:33:49.227899 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.227880 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 03 16:33:49.450424 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.450390 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" event={"ID":"cd9a1f99-6cd1-4665-819a-5f61ceb50998","Type":"ContainerStarted","Data":"eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083"} Aug 03 16:33:49.450573 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.450445 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" event={"ID":"cd9a1f99-6cd1-4665-819a-5f61ceb50998","Type":"ContainerStarted","Data":"8ab4838c3ffe63757e9d996d5776bbe159b4dd8021df8bd4ba152caf765a85b8"} Aug 03 16:33:49.473132 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.473086 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" podStartSLOduration=1.473073262 podStartE2EDuration="1.473073262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:33:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:33:49.470979488 +0000 UTC m=+1768.029062081" watchObservedRunningTime="2026-08-03 16:33:49.473073262 +0000 UTC m=+1768.031155846" Aug 03 16:33:49.978486 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:49.978452 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:50.982449 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:50.982421 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:51.460998 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:51.460964 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:51.461870 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:51.461853 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:33:52.815789 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.815767 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:52.868280 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868202 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-podinfo\") pod \"b2390e10-2101-4332-bee1-b4f17fd35c7a\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " Aug 03 16:33:52.868280 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868236 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-envoy\") pod \"b2390e10-2101-4332-bee1-b4f17fd35c7a\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " Aug 03 16:33:52.868280 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868266 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbx4s\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-kube-api-access-gbx4s\") pod \"b2390e10-2101-4332-bee1-b4f17fd35c7a\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " Aug 03 16:33:52.868515 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868299 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-token\") pod \"b2390e10-2101-4332-bee1-b4f17fd35c7a\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " Aug 03 16:33:52.868515 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868363 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-certs\") pod \"b2390e10-2101-4332-bee1-b4f17fd35c7a\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " Aug 03 16:33:52.868515 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868391 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-data\") pod \"b2390e10-2101-4332-bee1-b4f17fd35c7a\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " Aug 03 16:33:52.868515 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868427 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-credential-socket\") pod \"b2390e10-2101-4332-bee1-b4f17fd35c7a\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " Aug 03 16:33:52.868515 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868466 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-socket\") pod \"b2390e10-2101-4332-bee1-b4f17fd35c7a\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " Aug 03 16:33:52.868800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868523 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2390e10-2101-4332-bee1-b4f17fd35c7a-istiod-ca-cert\") pod \"b2390e10-2101-4332-bee1-b4f17fd35c7a\" (UID: \"b2390e10-2101-4332-bee1-b4f17fd35c7a\") " Aug 03 16:33:52.868800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868696 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2390e10-2101-4332-bee1-b4f17fd35c7a" (UID: "b2390e10-2101-4332-bee1-b4f17fd35c7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:33:52.868800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868764 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2390e10-2101-4332-bee1-b4f17fd35c7a" (UID: "b2390e10-2101-4332-bee1-b4f17fd35c7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:33:52.868800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868777 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2390e10-2101-4332-bee1-b4f17fd35c7a" (UID: "b2390e10-2101-4332-bee1-b4f17fd35c7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:33:52.869000 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868933 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2390e10-2101-4332-bee1-b4f17fd35c7a" (UID: "b2390e10-2101-4332-bee1-b4f17fd35c7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:33:52.869000 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868970 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:33:52.869000 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.868990 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:33:52.869121 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.869003 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:33:52.869121 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.869022 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2390e10-2101-4332-bee1-b4f17fd35c7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2390e10-2101-4332-bee1-b4f17fd35c7a" (UID: "b2390e10-2101-4332-bee1-b4f17fd35c7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:33:52.870576 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.870553 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2390e10-2101-4332-bee1-b4f17fd35c7a" (UID: "b2390e10-2101-4332-bee1-b4f17fd35c7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:33:52.870729 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.870648 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-kube-api-access-gbx4s" (OuterVolumeSpecName: "kube-api-access-gbx4s") pod "b2390e10-2101-4332-bee1-b4f17fd35c7a" (UID: "b2390e10-2101-4332-bee1-b4f17fd35c7a"). InnerVolumeSpecName "kube-api-access-gbx4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:33:52.870799 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.870779 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2390e10-2101-4332-bee1-b4f17fd35c7a" (UID: "b2390e10-2101-4332-bee1-b4f17fd35c7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:33:52.870852 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.870795 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2390e10-2101-4332-bee1-b4f17fd35c7a" (UID: "b2390e10-2101-4332-bee1-b4f17fd35c7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:33:52.970334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.970304 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:33:52.970334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.970329 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:33:52.970334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.970339 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2390e10-2101-4332-bee1-b4f17fd35c7a-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:33:52.970565 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.970347 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:33:52.970565 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.970355 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2390e10-2101-4332-bee1-b4f17fd35c7a-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:33:52.970565 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:52.970364 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbx4s\" (UniqueName: \"kubernetes.io/projected/b2390e10-2101-4332-bee1-b4f17fd35c7a-kube-api-access-gbx4s\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:33:53.473428 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.473395 2582 generic.go:356] "Generic (PLEG): container finished" podID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerID="41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca" exitCode=0 Aug 03 16:33:53.473606 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.473442 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" Aug 03 16:33:53.473606 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.473476 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" event={"ID":"b2390e10-2101-4332-bee1-b4f17fd35c7a","Type":"ContainerDied","Data":"41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca"} Aug 03 16:33:53.473606 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.473534 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf" event={"ID":"b2390e10-2101-4332-bee1-b4f17fd35c7a","Type":"ContainerDied","Data":"1e880cd166e4c27e47531d0a28ba3278f73fa3d09fdfc288fbf7b11ce5cddae0"} Aug 03 16:33:53.473606 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.473561 2582 scope.go:117] "RemoveContainer" containerID="41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca" Aug 03 16:33:53.487943 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.487912 2582 scope.go:117] "RemoveContainer" containerID="7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504" Aug 03 16:33:53.495855 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.495834 2582 scope.go:117] "RemoveContainer" containerID="41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca" Aug 03 16:33:53.496090 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:33:53.496071 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca\": container with ID starting with 41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca not found: ID does not exist" containerID="41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca" Aug 03 16:33:53.496136 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.496098 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca"} err="failed to get container status \"41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca\": rpc error: code = NotFound desc = could not find container \"41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca\": container with ID starting with 41ad295fff61be5a9fa83b1c2634e25654427c34f61d2b554fd7412ec9dfe6ca not found: ID does not exist" Aug 03 16:33:53.496136 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.496114 2582 scope.go:117] "RemoveContainer" containerID="7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504" Aug 03 16:33:53.496382 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:33:53.496362 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504\": container with ID starting with 7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504 not found: ID does not exist" containerID="7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504" Aug 03 16:33:53.496426 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.496388 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504"} err="failed to get container status \"7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504\": rpc error: code = NotFound desc = could not find container \"7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504\": container with ID starting with 7f390021bbbe41d8a8c1608954bad1b409424e628392c68a8038359b0f387504 not found: ID does not exist" Aug 03 16:33:53.499631 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.499602 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf"] Aug 03 16:33:53.504597 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.504577 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-77570ccf-openshift-default-6d4d84d776-rjjcf"] Aug 03 16:33:53.994978 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:33:53.994946 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" path="/var/lib/kubelet/pods/b2390e10-2101-4332-bee1-b4f17fd35c7a/volumes" Aug 03 16:34:03.531892 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:03.531856 2582 generic.go:356] "Generic (PLEG): container finished" podID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerID="eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083" exitCode=0 Aug 03 16:34:03.532282 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:03.531931 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" event={"ID":"cd9a1f99-6cd1-4665-819a-5f61ceb50998","Type":"ContainerDied","Data":"eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083"} Aug 03 16:34:03.532323 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:03.532297 2582 scope.go:117] "RemoveContainer" containerID="eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083" Aug 03 16:34:03.533281 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:03.533254 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:03.533380 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:03.533320 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:03.533380 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:03.533355 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:04.565089 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:04.565053 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" event={"ID":"cd9a1f99-6cd1-4665-819a-5f61ceb50998","Type":"ContainerStarted","Data":"bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d"} Aug 03 16:34:04.978281 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:04.978187 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:34:04.979287 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:04.979263 2582 patch_prober.go:28] interesting pod/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 03 16:34:04.979379 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:04.979332 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 03 16:34:05.978510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:05.978472 2582 patch_prober.go:28] interesting pod/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 03 16:34:05.978932 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:05.978555 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 03 16:34:06.461877 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:06.461848 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:34:06.978235 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:06.978194 2582 patch_prober.go:28] interesting pod/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 03 16:34:06.978416 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:06.978259 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 03 16:34:07.977964 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:07.977927 2582 patch_prober.go:28] interesting pod/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 03 16:34:07.978456 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:07.977989 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 03 16:34:08.978518 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:08.978477 2582 patch_prober.go:28] interesting pod/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Aug 03 16:34:08.979042 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:08.978544 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Aug 03 16:34:09.982736 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:09.982706 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:34:09.983758 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:09.983743 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:34:21.947427 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:21.947390 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g"] Aug 03 16:34:21.947942 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:21.947671 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" containerID="cri-o://bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d" gracePeriod=30 Aug 03 16:34:22.692130 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.692098 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5"] Aug 03 16:34:22.692552 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.692536 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" Aug 03 16:34:22.692656 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.692555 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" Aug 03 16:34:22.692656 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.692565 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" Aug 03 16:34:22.692656 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.692573 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" Aug 03 16:34:22.692814 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.692674 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" Aug 03 16:34:22.692814 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.692693 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2390e10-2101-4332-bee1-b4f17fd35c7a" containerName="istio-proxy" Aug 03 16:34:22.695387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.695369 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.698165 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.698144 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4f378709-openshift-default-dockercfg-d4mkr\"" Aug 03 16:34:22.708588 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.708563 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5"] Aug 03 16:34:22.844288 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.844242 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-podinfo\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.844472 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.844295 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-certs\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.844472 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.844326 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f67px\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-kube-api-access-f67px\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.844472 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.844363 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-data\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.844472 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.844447 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-socket\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.844610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.844481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-credential-socket\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.844610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.844506 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istiod-ca-cert\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.844610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.844563 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-envoy\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.844610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.844597 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-token\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.945984 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.945881 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-socket\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.945984 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.945931 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-credential-socket\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946230 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946060 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istiod-ca-cert\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946230 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946131 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-envoy\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946230 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946161 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-token\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946232 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-podinfo\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946261 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-certs\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946293 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f67px\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-kube-api-access-f67px\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946338 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-data\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946350 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-socket\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946638 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946402 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-credential-socket\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946709 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946686 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-data\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946809 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946790 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-certs\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.946874 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.946835 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istiod-ca-cert\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.948477 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.948454 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-envoy\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.948802 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.948784 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-podinfo\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.954450 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.954421 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-token\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:22.954602 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:22.954584 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f67px\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-kube-api-access-f67px\") pod \"e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:23.005844 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:23.005819 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:23.142289 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:23.142255 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5"] Aug 03 16:34:23.143011 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:34:23.142980 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac4fbc4_3046_4721_92d6_8999e08ea5ac.slice/crio-0af84c597fcb98deaa8908c9b223839fef9d59354c57da45d0c29c03a73f0989 WatchSource:0}: Error finding container 0af84c597fcb98deaa8908c9b223839fef9d59354c57da45d0c29c03a73f0989: Status 404 returned error can't find the container with id 0af84c597fcb98deaa8908c9b223839fef9d59354c57da45d0c29c03a73f0989 Aug 03 16:34:23.145136 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:23.145107 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:23.145208 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:23.145172 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:23.145246 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:23.145204 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:23.675274 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:23.675244 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" event={"ID":"3ac4fbc4-3046-4721-92d6-8999e08ea5ac","Type":"ContainerStarted","Data":"eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb"} Aug 03 16:34:23.675477 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:23.675301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" event={"ID":"3ac4fbc4-3046-4721-92d6-8999e08ea5ac","Type":"ContainerStarted","Data":"0af84c597fcb98deaa8908c9b223839fef9d59354c57da45d0c29c03a73f0989"} Aug 03 16:34:23.702387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:23.702336 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" podStartSLOduration=1.7023218500000001 podStartE2EDuration="1.70232185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:34:23.700704808 +0000 UTC m=+1802.258787393" watchObservedRunningTime="2026-08-03 16:34:23.70232185 +0000 UTC m=+1802.260404435" Aug 03 16:34:24.006488 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:24.006412 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:24.011867 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:24.011844 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:24.680783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:24.680754 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:24.681661 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:24.681646 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:24.985213 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:24.985137 2582 patch_prober.go:28] interesting pod/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 03 16:34:24.985213 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:24.985191 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 03 16:34:27.186482 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.186460 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:34:27.281194 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281158 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-certs\") pod \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " Aug 03 16:34:27.281370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281216 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-socket\") pod \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " Aug 03 16:34:27.281370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281241 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istiod-ca-cert\") pod \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " Aug 03 16:34:27.281370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281259 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-envoy\") pod \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " Aug 03 16:34:27.281370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281289 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-token\") pod \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " Aug 03 16:34:27.281370 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281357 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w7fg\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-kube-api-access-9w7fg\") pod \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " Aug 03 16:34:27.281742 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281381 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-podinfo\") pod \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " Aug 03 16:34:27.281742 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281410 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-credential-socket\") pod \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " Aug 03 16:34:27.281742 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281453 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-data\") pod \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\" (UID: \"cd9a1f99-6cd1-4665-819a-5f61ceb50998\") " Aug 03 16:34:27.281742 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281506 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd9a1f99-6cd1-4665-819a-5f61ceb50998" (UID: "cd9a1f99-6cd1-4665-819a-5f61ceb50998"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:34:27.281742 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281572 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd9a1f99-6cd1-4665-819a-5f61ceb50998" (UID: "cd9a1f99-6cd1-4665-819a-5f61ceb50998"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:34:27.281742 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281650 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd9a1f99-6cd1-4665-819a-5f61ceb50998" (UID: "cd9a1f99-6cd1-4665-819a-5f61ceb50998"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:34:27.282006 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281828 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd9a1f99-6cd1-4665-819a-5f61ceb50998" (UID: "cd9a1f99-6cd1-4665-819a-5f61ceb50998"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:34:27.282006 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281848 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:34:27.282006 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281876 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:34:27.282006 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281890 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:34:27.282006 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.281843 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd9a1f99-6cd1-4665-819a-5f61ceb50998" (UID: "cd9a1f99-6cd1-4665-819a-5f61ceb50998"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:34:27.283484 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.283458 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd9a1f99-6cd1-4665-819a-5f61ceb50998" (UID: "cd9a1f99-6cd1-4665-819a-5f61ceb50998"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:34:27.283593 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.283462 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd9a1f99-6cd1-4665-819a-5f61ceb50998" (UID: "cd9a1f99-6cd1-4665-819a-5f61ceb50998"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:34:27.283869 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.283852 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd9a1f99-6cd1-4665-819a-5f61ceb50998" (UID: "cd9a1f99-6cd1-4665-819a-5f61ceb50998"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:34:27.283975 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.283960 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-kube-api-access-9w7fg" (OuterVolumeSpecName: "kube-api-access-9w7fg") pod "cd9a1f99-6cd1-4665-819a-5f61ceb50998" (UID: "cd9a1f99-6cd1-4665-819a-5f61ceb50998"). InnerVolumeSpecName "kube-api-access-9w7fg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:34:27.382461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.382428 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:34:27.382461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.382458 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:34:27.382461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.382468 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:34:27.382720 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.382476 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:34:27.382720 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.382484 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w7fg\" (UniqueName: \"kubernetes.io/projected/cd9a1f99-6cd1-4665-819a-5f61ceb50998-kube-api-access-9w7fg\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:34:27.382720 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.382493 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd9a1f99-6cd1-4665-819a-5f61ceb50998-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:34:27.700455 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.700369 2582 generic.go:356] "Generic (PLEG): container finished" podID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerID="bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d" exitCode=0 Aug 03 16:34:27.700455 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.700412 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" Aug 03 16:34:27.700664 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.700451 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" event={"ID":"cd9a1f99-6cd1-4665-819a-5f61ceb50998","Type":"ContainerDied","Data":"bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d"} Aug 03 16:34:27.700664 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.700504 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g" event={"ID":"cd9a1f99-6cd1-4665-819a-5f61ceb50998","Type":"ContainerDied","Data":"8ab4838c3ffe63757e9d996d5776bbe159b4dd8021df8bd4ba152caf765a85b8"} Aug 03 16:34:27.700664 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.700533 2582 scope.go:117] "RemoveContainer" containerID="bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d" Aug 03 16:34:27.710035 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.710017 2582 scope.go:117] "RemoveContainer" containerID="eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083" Aug 03 16:34:27.717886 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.717869 2582 scope.go:117] "RemoveContainer" containerID="bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d" Aug 03 16:34:27.718116 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:34:27.718097 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d\": container with ID starting with bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d not found: ID does not exist" containerID="bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d" Aug 03 16:34:27.718164 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.718129 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d"} err="failed to get container status \"bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d\": rpc error: code = NotFound desc = could not find container \"bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d\": container with ID starting with bc2ef6857e3ddacda38ce2da2f69dfee73e7a93f1f816b9013e7fbdd9112342d not found: ID does not exist" Aug 03 16:34:27.718164 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.718152 2582 scope.go:117] "RemoveContainer" containerID="eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083" Aug 03 16:34:27.718395 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:34:27.718376 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083\": container with ID starting with eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083 not found: ID does not exist" containerID="eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083" Aug 03 16:34:27.718449 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.718399 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083"} err="failed to get container status \"eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083\": rpc error: code = NotFound desc = could not find container \"eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083\": container with ID starting with eea555762992f29b55945554f78ec8101292653655fc63c1d45651a1cb0d7083 not found: ID does not exist" Aug 03 16:34:27.723584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.723563 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g"] Aug 03 16:34:27.731842 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.731814 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-51e95abc-gw-openshift-default-568878d5b-r2h5g"] Aug 03 16:34:27.995710 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:27.995601 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" path="/var/lib/kubelet/pods/cd9a1f99-6cd1-4665-819a-5f61ceb50998/volumes" Aug 03 16:34:38.767082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:38.767047 2582 generic.go:356] "Generic (PLEG): container finished" podID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerID="eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb" exitCode=0 Aug 03 16:34:38.767082 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:38.767084 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" event={"ID":"3ac4fbc4-3046-4721-92d6-8999e08ea5ac","Type":"ContainerDied","Data":"eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb"} Aug 03 16:34:38.767603 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:38.767429 2582 scope.go:117] "RemoveContainer" containerID="eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb" Aug 03 16:34:38.768398 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:38.768374 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:38.768453 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:38.768435 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:38.768490 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:38.768466 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:39.681523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:39.681493 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:39.773689 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:39.773655 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" event={"ID":"3ac4fbc4-3046-4721-92d6-8999e08ea5ac","Type":"ContainerStarted","Data":"62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162"} Aug 03 16:34:40.006511 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:40.006431 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:40.007347 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:40.007326 2582 patch_prober.go:28] interesting pod/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Aug 03 16:34:40.007428 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:40.007366 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Aug 03 16:34:41.006970 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:41.006933 2582 patch_prober.go:28] interesting pod/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Aug 03 16:34:41.007363 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:41.006994 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Aug 03 16:34:42.006962 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:42.006928 2582 patch_prober.go:28] interesting pod/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Aug 03 16:34:42.007137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:42.006984 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Aug 03 16:34:43.007153 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:43.007117 2582 patch_prober.go:28] interesting pod/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Aug 03 16:34:43.007534 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:43.007181 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Aug 03 16:34:44.006832 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:44.006798 2582 patch_prober.go:28] interesting pod/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Aug 03 16:34:44.007013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:44.006857 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Aug 03 16:34:45.010234 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:45.010203 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:45.010692 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:45.010591 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:45.011258 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:45.011238 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:34:58.174700 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.174667 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5"] Aug 03 16:34:58.175271 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.174890 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" containerID="cri-o://62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162" gracePeriod=30 Aug 03 16:34:58.926075 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.926036 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk"] Aug 03 16:34:58.926998 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.926972 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" Aug 03 16:34:58.927150 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.927136 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" Aug 03 16:34:58.927260 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.927248 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" Aug 03 16:34:58.927353 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.927342 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" Aug 03 16:34:58.927553 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.927537 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" Aug 03 16:34:58.930172 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.930152 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:58.939157 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.939137 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-fed81e80-openshift-default-dockercfg-48zgp\"" Aug 03 16:34:58.945366 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:58.945319 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk"] Aug 03 16:34:59.093044 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.093008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-credential-socket\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.093233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.093054 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-data\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.093233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.093094 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-socket\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.093233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.093109 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-envoy\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.093233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.093132 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2155d924-aa28-484b-b74d-3f517216c3e5-istiod-ca-cert\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.093233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.093151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2155d924-aa28-484b-b74d-3f517216c3e5-istio-podinfo\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.093433 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.093261 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-istio-token\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.093433 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.093287 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffzwg\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-kube-api-access-ffzwg\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.093433 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.093313 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-certs\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.194880 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.194794 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2155d924-aa28-484b-b74d-3f517216c3e5-istiod-ca-cert\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.194880 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.194830 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2155d924-aa28-484b-b74d-3f517216c3e5-istio-podinfo\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.194880 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.194867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-istio-token\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.194893 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffzwg\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-kube-api-access-ffzwg\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.194915 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-certs\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.194952 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-credential-socket\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.194994 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-data\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.195038 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-socket\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.195060 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-envoy\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.195341 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-certs\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.195364 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-credential-socket\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195764 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.195435 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-data\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195764 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.195470 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-socket\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.195905 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.195882 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2155d924-aa28-484b-b74d-3f517216c3e5-istiod-ca-cert\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.197340 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.197323 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-envoy\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.197461 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.197444 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2155d924-aa28-484b-b74d-3f517216c3e5-istio-podinfo\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.207281 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.207258 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-istio-token\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.210450 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.210431 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffzwg\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-kube-api-access-ffzwg\") pod \"e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.250388 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.250356 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:34:59.403365 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.403341 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk"] Aug 03 16:34:59.404976 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:34:59.404955 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2155d924_aa28_484b_b74d_3f517216c3e5.slice/crio-c684a424c9473930bd68b278260a5b22cf9b9d4e74b05a830e28d397b49a78ee WatchSource:0}: Error finding container c684a424c9473930bd68b278260a5b22cf9b9d4e74b05a830e28d397b49a78ee: Status 404 returned error can't find the container with id c684a424c9473930bd68b278260a5b22cf9b9d4e74b05a830e28d397b49a78ee Aug 03 16:34:59.406959 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.406928 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:59.407011 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.406992 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:59.407049 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.407020 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:34:59.885278 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.885247 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" event={"ID":"2155d924-aa28-484b-b74d-3f517216c3e5","Type":"ContainerStarted","Data":"d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b"} Aug 03 16:34:59.885464 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.885301 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" event={"ID":"2155d924-aa28-484b-b74d-3f517216c3e5","Type":"ContainerStarted","Data":"c684a424c9473930bd68b278260a5b22cf9b9d4e74b05a830e28d397b49a78ee"} Aug 03 16:34:59.911285 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:34:59.911235 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" podStartSLOduration=1.911218772 podStartE2EDuration="1.911218772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:34:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:34:59.908900292 +0000 UTC m=+1838.466982878" watchObservedRunningTime="2026-08-03 16:34:59.911218772 +0000 UTC m=+1838.469301423" Aug 03 16:35:00.012396 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:00.012366 2582 patch_prober.go:28] interesting pod/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 03 16:35:00.012574 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:00.012421 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 03 16:35:00.250849 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:00.250758 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:00.255517 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:00.255490 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:00.890848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:00.890815 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:00.891675 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:00.891654 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:03.420051 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.420031 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:35:03.443355 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443274 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-data\") pod \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " Aug 03 16:35:03.443355 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443339 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-socket\") pod \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " Aug 03 16:35:03.443548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443368 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istiod-ca-cert\") pod \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " Aug 03 16:35:03.443548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443398 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-certs\") pod \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " Aug 03 16:35:03.443548 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443429 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f67px\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-kube-api-access-f67px\") pod \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " Aug 03 16:35:03.443744 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443536 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ac4fbc4-3046-4721-92d6-8999e08ea5ac" (UID: "3ac4fbc4-3046-4721-92d6-8999e08ea5ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:03.443744 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443589 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ac4fbc4-3046-4721-92d6-8999e08ea5ac" (UID: "3ac4fbc4-3046-4721-92d6-8999e08ea5ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:03.443744 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443599 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-token\") pod \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " Aug 03 16:35:03.443744 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443652 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-credential-socket\") pod \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " Aug 03 16:35:03.443744 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443685 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-podinfo\") pod \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " Aug 03 16:35:03.444003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443760 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ac4fbc4-3046-4721-92d6-8999e08ea5ac" (UID: "3ac4fbc4-3046-4721-92d6-8999e08ea5ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:35:03.444003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443802 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-envoy\") pod \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\" (UID: \"3ac4fbc4-3046-4721-92d6-8999e08ea5ac\") " Aug 03 16:35:03.444003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443870 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ac4fbc4-3046-4721-92d6-8999e08ea5ac" (UID: "3ac4fbc4-3046-4721-92d6-8999e08ea5ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:03.444003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.443922 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ac4fbc4-3046-4721-92d6-8999e08ea5ac" (UID: "3ac4fbc4-3046-4721-92d6-8999e08ea5ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:03.444214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.444113 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:03.444214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.444128 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:03.444214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.444141 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:03.444214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.444154 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:03.444214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.444166 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:03.447950 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.447909 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ac4fbc4-3046-4721-92d6-8999e08ea5ac" (UID: "3ac4fbc4-3046-4721-92d6-8999e08ea5ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:03.448083 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.448034 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-kube-api-access-f67px" (OuterVolumeSpecName: "kube-api-access-f67px") pod "3ac4fbc4-3046-4721-92d6-8999e08ea5ac" (UID: "3ac4fbc4-3046-4721-92d6-8999e08ea5ac"). InnerVolumeSpecName "kube-api-access-f67px". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:35:03.448547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.448524 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ac4fbc4-3046-4721-92d6-8999e08ea5ac" (UID: "3ac4fbc4-3046-4721-92d6-8999e08ea5ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:35:03.450315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.450269 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ac4fbc4-3046-4721-92d6-8999e08ea5ac" (UID: "3ac4fbc4-3046-4721-92d6-8999e08ea5ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:35:03.545665 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.545629 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f67px\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-kube-api-access-f67px\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:03.545665 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.545662 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:03.545665 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.545671 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:03.545903 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.545680 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ac4fbc4-3046-4721-92d6-8999e08ea5ac-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:03.910508 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.910477 2582 generic.go:356] "Generic (PLEG): container finished" podID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerID="62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162" exitCode=0 Aug 03 16:35:03.910713 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.910520 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" Aug 03 16:35:03.910713 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.910560 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" event={"ID":"3ac4fbc4-3046-4721-92d6-8999e08ea5ac","Type":"ContainerDied","Data":"62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162"} Aug 03 16:35:03.910713 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.910613 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5" event={"ID":"3ac4fbc4-3046-4721-92d6-8999e08ea5ac","Type":"ContainerDied","Data":"0af84c597fcb98deaa8908c9b223839fef9d59354c57da45d0c29c03a73f0989"} Aug 03 16:35:03.910713 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.910674 2582 scope.go:117] "RemoveContainer" containerID="62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162" Aug 03 16:35:03.921667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.921648 2582 scope.go:117] "RemoveContainer" containerID="eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb" Aug 03 16:35:03.930063 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.930045 2582 scope.go:117] "RemoveContainer" containerID="62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162" Aug 03 16:35:03.930316 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:35:03.930293 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162\": container with ID starting with 62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162 not found: ID does not exist" containerID="62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162" Aug 03 16:35:03.930376 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.930323 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162"} err="failed to get container status \"62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162\": rpc error: code = NotFound desc = could not find container \"62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162\": container with ID starting with 62004ed80f109e2782466e5800f08d01ab6285920f5aa75d87c5dc8bc1135162 not found: ID does not exist" Aug 03 16:35:03.930376 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.930340 2582 scope.go:117] "RemoveContainer" containerID="eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb" Aug 03 16:35:03.930546 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:35:03.930531 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb\": container with ID starting with eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb not found: ID does not exist" containerID="eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb" Aug 03 16:35:03.930585 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.930550 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb"} err="failed to get container status \"eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb\": rpc error: code = NotFound desc = could not find container \"eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb\": container with ID starting with eaa9958ce5a58defb000aad9e9d2ffc9b10607d65ec2b5beffde534b19b79ceb not found: ID does not exist" Aug 03 16:35:03.941125 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.939501 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5"] Aug 03 16:35:03.946858 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.946833 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4f378709-openshift-default-54cc8f86b4-l98t5"] Aug 03 16:35:03.995184 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:03.995151 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" path="/var/lib/kubelet/pods/3ac4fbc4-3046-4721-92d6-8999e08ea5ac/volumes" Aug 03 16:35:14.987233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:14.987200 2582 generic.go:356] "Generic (PLEG): container finished" podID="2155d924-aa28-484b-b74d-3f517216c3e5" containerID="d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b" exitCode=0 Aug 03 16:35:14.987708 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:14.987272 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" event={"ID":"2155d924-aa28-484b-b74d-3f517216c3e5","Type":"ContainerDied","Data":"d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b"} Aug 03 16:35:14.987708 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:14.987690 2582 scope.go:117] "RemoveContainer" containerID="d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b" Aug 03 16:35:14.988692 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:14.988664 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:35:14.988814 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:14.988745 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:35:14.988814 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:14.988792 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:35:15.891654 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:15.891600 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:15.995450 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:15.995413 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" event={"ID":"2155d924-aa28-484b-b74d-3f517216c3e5","Type":"ContainerStarted","Data":"f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7"} Aug 03 16:35:16.251518 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:16.251420 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:16.252611 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:16.252575 2582 patch_prober.go:28] interesting pod/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Aug 03 16:35:16.252758 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:16.252653 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Aug 03 16:35:17.251177 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:17.251142 2582 patch_prober.go:28] interesting pod/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Aug 03 16:35:17.251786 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:17.251756 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Aug 03 16:35:18.251244 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:18.251200 2582 patch_prober.go:28] interesting pod/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Aug 03 16:35:18.251782 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:18.251272 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Aug 03 16:35:19.251208 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:19.251156 2582 patch_prober.go:28] interesting pod/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Aug 03 16:35:19.251425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:19.251229 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Aug 03 16:35:20.251386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:20.251335 2582 patch_prober.go:28] interesting pod/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Aug 03 16:35:20.251632 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:20.251409 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Aug 03 16:35:21.255173 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:21.255145 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:21.255543 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:21.255514 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:21.256037 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:21.256021 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:33.449361 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:33.449322 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk"] Aug 03 16:35:33.450017 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:33.449570 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" containerID="cri-o://f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7" gracePeriod=30 Aug 03 16:35:36.258200 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:36.258162 2582 patch_prober.go:28] interesting pod/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 03 16:35:36.258690 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:36.258225 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 03 16:35:38.690911 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.690890 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:38.786717 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.786683 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-envoy\") pod \"2155d924-aa28-484b-b74d-3f517216c3e5\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " Aug 03 16:35:38.786891 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.786725 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-certs\") pod \"2155d924-aa28-484b-b74d-3f517216c3e5\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " Aug 03 16:35:38.786891 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.786740 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-socket\") pod \"2155d924-aa28-484b-b74d-3f517216c3e5\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " Aug 03 16:35:38.786891 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.786783 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2155d924-aa28-484b-b74d-3f517216c3e5-istio-podinfo\") pod \"2155d924-aa28-484b-b74d-3f517216c3e5\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " Aug 03 16:35:38.786891 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.786811 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2155d924-aa28-484b-b74d-3f517216c3e5-istiod-ca-cert\") pod \"2155d924-aa28-484b-b74d-3f517216c3e5\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " Aug 03 16:35:38.786891 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.786829 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-credential-socket\") pod \"2155d924-aa28-484b-b74d-3f517216c3e5\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " Aug 03 16:35:38.787146 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.786940 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffzwg\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-kube-api-access-ffzwg\") pod \"2155d924-aa28-484b-b74d-3f517216c3e5\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " Aug 03 16:35:38.787146 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.786989 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-istio-token\") pod \"2155d924-aa28-484b-b74d-3f517216c3e5\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " Aug 03 16:35:38.787146 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787024 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2155d924-aa28-484b-b74d-3f517216c3e5" (UID: "2155d924-aa28-484b-b74d-3f517216c3e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:38.787146 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787047 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-data\") pod \"2155d924-aa28-484b-b74d-3f517216c3e5\" (UID: \"2155d924-aa28-484b-b74d-3f517216c3e5\") " Aug 03 16:35:38.787346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787167 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2155d924-aa28-484b-b74d-3f517216c3e5" (UID: "2155d924-aa28-484b-b74d-3f517216c3e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:38.787346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787179 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2155d924-aa28-484b-b74d-3f517216c3e5" (UID: "2155d924-aa28-484b-b74d-3f517216c3e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:38.787346 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787329 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-data" (OuterVolumeSpecName: "istio-data") pod "2155d924-aa28-484b-b74d-3f517216c3e5" (UID: "2155d924-aa28-484b-b74d-3f517216c3e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:38.787479 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787288 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2155d924-aa28-484b-b74d-3f517216c3e5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2155d924-aa28-484b-b74d-3f517216c3e5" (UID: "2155d924-aa28-484b-b74d-3f517216c3e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:35:38.787479 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787449 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:38.787479 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787467 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:38.787610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787481 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:38.787610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.787493 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:38.789020 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.788997 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2155d924-aa28-484b-b74d-3f517216c3e5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2155d924-aa28-484b-b74d-3f517216c3e5" (UID: "2155d924-aa28-484b-b74d-3f517216c3e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:35:38.789169 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.789139 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-kube-api-access-ffzwg" (OuterVolumeSpecName: "kube-api-access-ffzwg") pod "2155d924-aa28-484b-b74d-3f517216c3e5" (UID: "2155d924-aa28-484b-b74d-3f517216c3e5"). InnerVolumeSpecName "kube-api-access-ffzwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:35:38.789415 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.789396 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2155d924-aa28-484b-b74d-3f517216c3e5" (UID: "2155d924-aa28-484b-b74d-3f517216c3e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:35:38.789514 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.789434 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-istio-token" (OuterVolumeSpecName: "istio-token") pod "2155d924-aa28-484b-b74d-3f517216c3e5" (UID: "2155d924-aa28-484b-b74d-3f517216c3e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:35:38.888305 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.888268 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2155d924-aa28-484b-b74d-3f517216c3e5-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:38.888305 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.888296 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffzwg\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-kube-api-access-ffzwg\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:38.888305 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.888305 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2155d924-aa28-484b-b74d-3f517216c3e5-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:38.888305 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.888313 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2155d924-aa28-484b-b74d-3f517216c3e5-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:38.888553 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:38.888321 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2155d924-aa28-484b-b74d-3f517216c3e5-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:35:39.126612 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.126526 2582 generic.go:356] "Generic (PLEG): container finished" podID="2155d924-aa28-484b-b74d-3f517216c3e5" containerID="f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7" exitCode=0 Aug 03 16:35:39.126612 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.126576 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" Aug 03 16:35:39.126861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.126631 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" event={"ID":"2155d924-aa28-484b-b74d-3f517216c3e5","Type":"ContainerDied","Data":"f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7"} Aug 03 16:35:39.126861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.126695 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk" event={"ID":"2155d924-aa28-484b-b74d-3f517216c3e5","Type":"ContainerDied","Data":"c684a424c9473930bd68b278260a5b22cf9b9d4e74b05a830e28d397b49a78ee"} Aug 03 16:35:39.126861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.126728 2582 scope.go:117] "RemoveContainer" containerID="f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7" Aug 03 16:35:39.136086 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.136068 2582 scope.go:117] "RemoveContainer" containerID="d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b" Aug 03 16:35:39.145394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.145375 2582 scope.go:117] "RemoveContainer" containerID="f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7" Aug 03 16:35:39.145662 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:35:39.145641 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7\": container with ID starting with f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7 not found: ID does not exist" containerID="f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7" Aug 03 16:35:39.145736 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.145670 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7"} err="failed to get container status \"f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7\": rpc error: code = NotFound desc = could not find container \"f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7\": container with ID starting with f1dd4a441f2d27750921208940e99191014467245f3163f5a744bc16399776f7 not found: ID does not exist" Aug 03 16:35:39.145736 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.145687 2582 scope.go:117] "RemoveContainer" containerID="d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b" Aug 03 16:35:39.145942 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:35:39.145923 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b\": container with ID starting with d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b not found: ID does not exist" containerID="d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b" Aug 03 16:35:39.146005 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.145948 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b"} err="failed to get container status \"d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b\": rpc error: code = NotFound desc = could not find container \"d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b\": container with ID starting with d62f8b2887dd28302cf29432c252a518ffff9290894968c9255355ac23408e8b not found: ID does not exist" Aug 03 16:35:39.150556 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.150537 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk"] Aug 03 16:35:39.154820 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.154791 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-fed81e80-openshift-default-f58979c44-wrwnk"] Aug 03 16:35:39.995770 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:35:39.995738 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" path="/var/lib/kubelet/pods/2155d924-aa28-484b-b74d-3f517216c3e5/volumes" Aug 03 16:37:32.471979 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.471900 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq"] Aug 03 16:37:32.472561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472473 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" Aug 03 16:37:32.472561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472491 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" Aug 03 16:37:32.472561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472503 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" Aug 03 16:37:32.472561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472511 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" Aug 03 16:37:32.472561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472524 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" Aug 03 16:37:32.472561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472533 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" Aug 03 16:37:32.472561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472546 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" Aug 03 16:37:32.472561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472554 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" Aug 03 16:37:32.472984 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472678 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" Aug 03 16:37:32.472984 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472692 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ac4fbc4-3046-4721-92d6-8999e08ea5ac" containerName="istio-proxy" Aug 03 16:37:32.472984 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472702 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" Aug 03 16:37:32.472984 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472715 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd9a1f99-6cd1-4665-819a-5f61ceb50998" containerName="istio-proxy" Aug 03 16:37:32.472984 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.472725 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2155d924-aa28-484b-b74d-3f517216c3e5" containerName="istio-proxy" Aug 03 16:37:32.474899 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.474880 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.477788 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.477771 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-de4e0ed8-openshift-default-dockercfg-7fsws\"" Aug 03 16:37:32.493210 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493185 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq"] Aug 03 16:37:32.493432 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-socket\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.493490 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493447 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-podinfo\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.493490 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493473 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-credential-socket\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.493605 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493583 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-data\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.493680 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493651 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9401fb5-8621-45dc-93c1-71b52d627e79-istiod-ca-cert\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.493743 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493703 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-certs\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.493798 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493742 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-token\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.493861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493793 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmcqv\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-kube-api-access-zmcqv\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.493861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.493821 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-envoy\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595014 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.594976 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9401fb5-8621-45dc-93c1-71b52d627e79-istiod-ca-cert\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595199 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595022 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-certs\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595199 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-token\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595199 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595075 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmcqv\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-kube-api-access-zmcqv\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595199 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595094 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-envoy\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595199 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595151 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-socket\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595199 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595176 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-podinfo\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595518 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595199 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-credential-socket\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595518 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595254 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-data\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595518 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595440 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-certs\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595695 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595548 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-socket\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595695 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595679 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-data\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595813 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595720 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-credential-socket\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.595813 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.595804 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9401fb5-8621-45dc-93c1-71b52d627e79-istiod-ca-cert\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.597528 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.597507 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-envoy\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.597630 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.597601 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-podinfo\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.607098 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.607074 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-token\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.607282 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.607260 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmcqv\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-kube-api-access-zmcqv\") pod \"e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.785278 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.785241 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:32.925237 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.925211 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq"] Aug 03 16:37:32.926423 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:37:32.926397 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9401fb5_8621_45dc_93c1_71b52d627e79.slice/crio-927bbbea95214d308ff14ad637b61ac9067d978e2ca4068d2eee3d325f5033de WatchSource:0}: Error finding container 927bbbea95214d308ff14ad637b61ac9067d978e2ca4068d2eee3d325f5033de: Status 404 returned error can't find the container with id 927bbbea95214d308ff14ad637b61ac9067d978e2ca4068d2eee3d325f5033de Aug 03 16:37:32.928305 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.928278 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:37:32.928360 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.928335 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:37:32.928402 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:32.928363 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:37:33.758746 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:33.758714 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" event={"ID":"f9401fb5-8621-45dc-93c1-71b52d627e79","Type":"ContainerStarted","Data":"01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921"} Aug 03 16:37:33.759234 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:33.758802 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" event={"ID":"f9401fb5-8621-45dc-93c1-71b52d627e79","Type":"ContainerStarted","Data":"927bbbea95214d308ff14ad637b61ac9067d978e2ca4068d2eee3d325f5033de"} Aug 03 16:37:33.783335 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:33.783286 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" podStartSLOduration=1.7832718600000002 podStartE2EDuration="1.78327186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:37:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:37:33.780903201 +0000 UTC m=+1992.338985796" watchObservedRunningTime="2026-08-03 16:37:33.78327186 +0000 UTC m=+1992.341354445" Aug 03 16:37:33.785839 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:33.785809 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:33.790648 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:33.790613 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:34.764403 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:34.764375 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:34.765439 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:34.765421 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:48.855486 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:48.855452 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerID="01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921" exitCode=0 Aug 03 16:37:48.855957 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:48.855524 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" event={"ID":"f9401fb5-8621-45dc-93c1-71b52d627e79","Type":"ContainerDied","Data":"01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921"} Aug 03 16:37:48.855957 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:48.855901 2582 scope.go:117] "RemoveContainer" containerID="01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921" Aug 03 16:37:48.856920 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:48.856895 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:37:48.856998 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:48.856974 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:37:48.857034 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:48.857019 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:37:49.764468 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:49.764430 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:49.865955 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:49.865918 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" event={"ID":"f9401fb5-8621-45dc-93c1-71b52d627e79","Type":"ContainerStarted","Data":"0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f"} Aug 03 16:37:50.786054 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:50.786012 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:50.787059 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:50.787037 2582 patch_prober.go:28] interesting pod/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Aug 03 16:37:50.787131 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:50.787111 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Aug 03 16:37:51.786499 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:51.786464 2582 patch_prober.go:28] interesting pod/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Aug 03 16:37:51.786918 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:51.786517 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Aug 03 16:37:52.786325 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:52.786284 2582 patch_prober.go:28] interesting pod/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Aug 03 16:37:52.786511 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:52.786351 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Aug 03 16:37:53.786337 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:53.786300 2582 patch_prober.go:28] interesting pod/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Aug 03 16:37:53.786760 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:53.786355 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Aug 03 16:37:54.789282 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:54.789251 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:54.789665 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:54.789580 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:37:54.790276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:37:54.790262 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:38:25.803438 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:25.803403 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq"] Aug 03 16:38:25.803953 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:25.803653 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" containerID="cri-o://0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f" gracePeriod=30 Aug 03 16:38:31.045436 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.045411 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:38:31.081705 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.081608 2582 generic.go:356] "Generic (PLEG): container finished" podID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerID="0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f" exitCode=0 Aug 03 16:38:31.081705 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.081657 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" event={"ID":"f9401fb5-8621-45dc-93c1-71b52d627e79","Type":"ContainerDied","Data":"0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f"} Aug 03 16:38:31.081705 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.081679 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" Aug 03 16:38:31.081941 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.081719 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq" event={"ID":"f9401fb5-8621-45dc-93c1-71b52d627e79","Type":"ContainerDied","Data":"927bbbea95214d308ff14ad637b61ac9067d978e2ca4068d2eee3d325f5033de"} Aug 03 16:38:31.081941 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.081756 2582 scope.go:117] "RemoveContainer" containerID="0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f" Aug 03 16:38:31.090741 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.090716 2582 scope.go:117] "RemoveContainer" containerID="01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921" Aug 03 16:38:31.098435 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.098417 2582 scope.go:117] "RemoveContainer" containerID="0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f" Aug 03 16:38:31.098671 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:38:31.098655 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f\": container with ID starting with 0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f not found: ID does not exist" containerID="0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f" Aug 03 16:38:31.098718 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.098680 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f"} err="failed to get container status \"0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f\": rpc error: code = NotFound desc = could not find container \"0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f\": container with ID starting with 0eb19cf94d08c312ed71616f51764d4206b9780f2ac931acedebe2760620b97f not found: ID does not exist" Aug 03 16:38:31.098718 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.098695 2582 scope.go:117] "RemoveContainer" containerID="01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921" Aug 03 16:38:31.098891 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:38:31.098876 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921\": container with ID starting with 01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921 not found: ID does not exist" containerID="01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921" Aug 03 16:38:31.098930 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.098894 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921"} err="failed to get container status \"01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921\": rpc error: code = NotFound desc = could not find container \"01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921\": container with ID starting with 01bede98aa02faab749e05e9ad68ab8efbd1ed8a8762d54a59796dd676c66921 not found: ID does not exist" Aug 03 16:38:31.163257 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163227 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-socket\") pod \"f9401fb5-8621-45dc-93c1-71b52d627e79\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " Aug 03 16:38:31.163432 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163271 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-data\") pod \"f9401fb5-8621-45dc-93c1-71b52d627e79\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " Aug 03 16:38:31.163432 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163286 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-certs\") pod \"f9401fb5-8621-45dc-93c1-71b52d627e79\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " Aug 03 16:38:31.163432 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163330 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-envoy\") pod \"f9401fb5-8621-45dc-93c1-71b52d627e79\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " Aug 03 16:38:31.163432 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163361 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-credential-socket\") pod \"f9401fb5-8621-45dc-93c1-71b52d627e79\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " Aug 03 16:38:31.163651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163433 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-podinfo\") pod \"f9401fb5-8621-45dc-93c1-71b52d627e79\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " Aug 03 16:38:31.163651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163468 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmcqv\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-kube-api-access-zmcqv\") pod \"f9401fb5-8621-45dc-93c1-71b52d627e79\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " Aug 03 16:38:31.163651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163517 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-token\") pod \"f9401fb5-8621-45dc-93c1-71b52d627e79\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " Aug 03 16:38:31.163651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163546 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9401fb5-8621-45dc-93c1-71b52d627e79-istiod-ca-cert\") pod \"f9401fb5-8621-45dc-93c1-71b52d627e79\" (UID: \"f9401fb5-8621-45dc-93c1-71b52d627e79\") " Aug 03 16:38:31.163651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163552 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f9401fb5-8621-45dc-93c1-71b52d627e79" (UID: "f9401fb5-8621-45dc-93c1-71b52d627e79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:38:31.163651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163593 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-data" (OuterVolumeSpecName: "istio-data") pod "f9401fb5-8621-45dc-93c1-71b52d627e79" (UID: "f9401fb5-8621-45dc-93c1-71b52d627e79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:38:31.163651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163611 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f9401fb5-8621-45dc-93c1-71b52d627e79" (UID: "f9401fb5-8621-45dc-93c1-71b52d627e79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:38:31.164034 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163826 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f9401fb5-8621-45dc-93c1-71b52d627e79" (UID: "f9401fb5-8621-45dc-93c1-71b52d627e79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:38:31.164034 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163846 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:38:31.164034 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163865 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:38:31.164034 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.163879 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:38:31.164194 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.164030 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9401fb5-8621-45dc-93c1-71b52d627e79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f9401fb5-8621-45dc-93c1-71b52d627e79" (UID: "f9401fb5-8621-45dc-93c1-71b52d627e79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:38:31.165672 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.165649 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-kube-api-access-zmcqv" (OuterVolumeSpecName: "kube-api-access-zmcqv") pod "f9401fb5-8621-45dc-93c1-71b52d627e79" (UID: "f9401fb5-8621-45dc-93c1-71b52d627e79"). InnerVolumeSpecName "kube-api-access-zmcqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:38:31.165799 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.165699 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f9401fb5-8621-45dc-93c1-71b52d627e79" (UID: "f9401fb5-8621-45dc-93c1-71b52d627e79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:38:31.165946 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.165917 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f9401fb5-8621-45dc-93c1-71b52d627e79" (UID: "f9401fb5-8621-45dc-93c1-71b52d627e79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:38:31.166042 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.165990 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-token" (OuterVolumeSpecName: "istio-token") pod "f9401fb5-8621-45dc-93c1-71b52d627e79" (UID: "f9401fb5-8621-45dc-93c1-71b52d627e79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:38:31.264928 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.264890 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:38:31.264928 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.264922 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9401fb5-8621-45dc-93c1-71b52d627e79-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:38:31.264928 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.264932 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:38:31.265157 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.264939 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9401fb5-8621-45dc-93c1-71b52d627e79-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:38:31.265157 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.264948 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9401fb5-8621-45dc-93c1-71b52d627e79-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:38:31.265157 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.264957 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmcqv\" (UniqueName: \"kubernetes.io/projected/f9401fb5-8621-45dc-93c1-71b52d627e79-kube-api-access-zmcqv\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:38:31.405545 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.405516 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq"] Aug 03 16:38:31.411462 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.411434 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-de4e0ed8-openshift-default-74748bff5b-jcmdq"] Aug 03 16:38:31.995085 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:38:31.995050 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" path="/var/lib/kubelet/pods/f9401fb5-8621-45dc-93c1-71b52d627e79/volumes" Aug 03 16:39:08.169912 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.169874 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8"] Aug 03 16:39:08.170531 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.170158 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" podUID="f24fab85-6f74-42d9-a16f-e154055e18ca" containerName="manager" containerID="cri-o://592de2ef058bb8638dbe57bff3e9d555aed3d31533c30b92e58f972ca8857af2" gracePeriod=10 Aug 03 16:39:08.299312 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.299282 2582 generic.go:356] "Generic (PLEG): container finished" podID="f24fab85-6f74-42d9-a16f-e154055e18ca" containerID="592de2ef058bb8638dbe57bff3e9d555aed3d31533c30b92e58f972ca8857af2" exitCode=0 Aug 03 16:39:08.299448 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.299359 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" event={"ID":"f24fab85-6f74-42d9-a16f-e154055e18ca","Type":"ContainerDied","Data":"592de2ef058bb8638dbe57bff3e9d555aed3d31533c30b92e58f972ca8857af2"} Aug 03 16:39:08.419523 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.419501 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:39:08.510269 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.510186 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24fab85-6f74-42d9-a16f-e154055e18ca-extensions-socket-volume\") pod \"f24fab85-6f74-42d9-a16f-e154055e18ca\" (UID: \"f24fab85-6f74-42d9-a16f-e154055e18ca\") " Aug 03 16:39:08.510441 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.510338 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44qmd\" (UniqueName: \"kubernetes.io/projected/f24fab85-6f74-42d9-a16f-e154055e18ca-kube-api-access-44qmd\") pod \"f24fab85-6f74-42d9-a16f-e154055e18ca\" (UID: \"f24fab85-6f74-42d9-a16f-e154055e18ca\") " Aug 03 16:39:08.510606 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.510582 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f24fab85-6f74-42d9-a16f-e154055e18ca-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f24fab85-6f74-42d9-a16f-e154055e18ca" (UID: "f24fab85-6f74-42d9-a16f-e154055e18ca"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:39:08.512267 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.512247 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f24fab85-6f74-42d9-a16f-e154055e18ca-kube-api-access-44qmd" (OuterVolumeSpecName: "kube-api-access-44qmd") pod "f24fab85-6f74-42d9-a16f-e154055e18ca" (UID: "f24fab85-6f74-42d9-a16f-e154055e18ca"). InnerVolumeSpecName "kube-api-access-44qmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:39:08.611084 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.611047 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44qmd\" (UniqueName: \"kubernetes.io/projected/f24fab85-6f74-42d9-a16f-e154055e18ca-kube-api-access-44qmd\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:39:08.611084 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:08.611077 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24fab85-6f74-42d9-a16f-e154055e18ca-extensions-socket-volume\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:39:09.307307 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:09.307283 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" Aug 03 16:39:09.307779 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:09.307321 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8" event={"ID":"f24fab85-6f74-42d9-a16f-e154055e18ca","Type":"ContainerDied","Data":"6f8394d163f62a33b14a64c7e1ead3f56b880abaac15bf35046cc311c4101030"} Aug 03 16:39:09.307779 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:09.307382 2582 scope.go:117] "RemoveContainer" containerID="592de2ef058bb8638dbe57bff3e9d555aed3d31533c30b92e58f972ca8857af2" Aug 03 16:39:09.336948 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:09.336915 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8"] Aug 03 16:39:09.340307 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:09.340286 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-xd6f8"] Aug 03 16:39:09.996030 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:39:09.995986 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f24fab85-6f74-42d9-a16f-e154055e18ca" path="/var/lib/kubelet/pods/f24fab85-6f74-42d9-a16f-e154055e18ca/volumes" Aug 03 16:40:23.815513 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.815430 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d"] Aug 03 16:40:23.816013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.815805 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f24fab85-6f74-42d9-a16f-e154055e18ca" containerName="manager" Aug 03 16:40:23.816013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.815815 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f24fab85-6f74-42d9-a16f-e154055e18ca" containerName="manager" Aug 03 16:40:23.816013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.815824 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" Aug 03 16:40:23.816013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.815830 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" Aug 03 16:40:23.816013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.815837 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" Aug 03 16:40:23.816013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.815843 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" Aug 03 16:40:23.816013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.815906 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" Aug 03 16:40:23.816013 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.815917 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f24fab85-6f74-42d9-a16f-e154055e18ca" containerName="manager" Aug 03 16:40:23.818147 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.818131 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:23.821262 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.821247 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lwnjp\"" Aug 03 16:40:23.829764 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.829736 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d"] Aug 03 16:40:23.876321 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.876292 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ggdm\" (UniqueName: \"kubernetes.io/projected/e404b6bd-15b3-4269-921b-02e6dbbda707-kube-api-access-9ggdm\") pod \"kuadrant-operator-controller-manager-68cf589589-mg52d\" (UID: \"e404b6bd-15b3-4269-921b-02e6dbbda707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:23.876478 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.876333 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e404b6bd-15b3-4269-921b-02e6dbbda707-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-mg52d\" (UID: \"e404b6bd-15b3-4269-921b-02e6dbbda707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:23.977080 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.977038 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ggdm\" (UniqueName: \"kubernetes.io/projected/e404b6bd-15b3-4269-921b-02e6dbbda707-kube-api-access-9ggdm\") pod \"kuadrant-operator-controller-manager-68cf589589-mg52d\" (UID: \"e404b6bd-15b3-4269-921b-02e6dbbda707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:23.977280 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.977092 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e404b6bd-15b3-4269-921b-02e6dbbda707-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-mg52d\" (UID: \"e404b6bd-15b3-4269-921b-02e6dbbda707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:23.977478 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.977457 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e404b6bd-15b3-4269-921b-02e6dbbda707-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-mg52d\" (UID: \"e404b6bd-15b3-4269-921b-02e6dbbda707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:23.985780 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:23.985755 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ggdm\" (UniqueName: \"kubernetes.io/projected/e404b6bd-15b3-4269-921b-02e6dbbda707-kube-api-access-9ggdm\") pod \"kuadrant-operator-controller-manager-68cf589589-mg52d\" (UID: \"e404b6bd-15b3-4269-921b-02e6dbbda707\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:24.129307 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:24.129196 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:24.267267 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:24.267241 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d"] Aug 03 16:40:24.272433 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:40:24.271748 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode404b6bd_15b3_4269_921b_02e6dbbda707.slice/crio-33a042c3e1b1957bb563170701aa4518427c5b4b9dd56d6490bfc230c9d3442a WatchSource:0}: Error finding container 33a042c3e1b1957bb563170701aa4518427c5b4b9dd56d6490bfc230c9d3442a: Status 404 returned error can't find the container with id 33a042c3e1b1957bb563170701aa4518427c5b4b9dd56d6490bfc230c9d3442a Aug 03 16:40:24.705413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:24.705381 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" event={"ID":"e404b6bd-15b3-4269-921b-02e6dbbda707","Type":"ContainerStarted","Data":"a93d6d266ff54edf7db3795d6b52059d89069411229dc2548113210c3cb6728e"} Aug 03 16:40:24.705602 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:24.705522 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:24.705602 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:24.705569 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" event={"ID":"e404b6bd-15b3-4269-921b-02e6dbbda707","Type":"ContainerStarted","Data":"33a042c3e1b1957bb563170701aa4518427c5b4b9dd56d6490bfc230c9d3442a"} Aug 03 16:40:24.729841 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:24.729786 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" podStartSLOduration=1.729768835 podStartE2EDuration="1.729768835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:40:24.727881388 +0000 UTC m=+2163.285963974" watchObservedRunningTime="2026-08-03 16:40:24.729768835 +0000 UTC m=+2163.287851419" Aug 03 16:40:35.712426 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:35.712399 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-mg52d" Aug 03 16:40:42.579276 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.579206 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc"] Aug 03 16:40:42.579866 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.579846 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9401fb5-8621-45dc-93c1-71b52d627e79" containerName="istio-proxy" Aug 03 16:40:42.582940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.582921 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.586156 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.586137 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-65805bcc-openshift-default-dockercfg-ftrbs\"" Aug 03 16:40:42.596513 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.596492 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc"] Aug 03 16:40:42.752394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.752295 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-credential-socket\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.752394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.752339 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-certs\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.752394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.752384 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-istio-token\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.752674 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.752413 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4371e0f4-8593-40eb-a50f-41a984d70701-istiod-ca-cert\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.752674 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.752438 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-data\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.752674 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.752481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-socket\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.752674 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.752517 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-envoy\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.752674 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.752579 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh7qc\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-kube-api-access-nh7qc\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.752674 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.752641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4371e0f4-8593-40eb-a50f-41a984d70701-istio-podinfo\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853312 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853278 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-credential-socket\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853312 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853314 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-certs\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853352 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-istio-token\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853375 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4371e0f4-8593-40eb-a50f-41a984d70701-istiod-ca-cert\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853397 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-data\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853417 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-socket\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853449 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-envoy\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853491 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh7qc\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-kube-api-access-nh7qc\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853524 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4371e0f4-8593-40eb-a50f-41a984d70701-istio-podinfo\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853929 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853826 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-certs\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853929 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853890 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-data\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.853929 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.853905 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-socket\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.854154 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.854129 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-credential-socket\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.854268 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.854206 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4371e0f4-8593-40eb-a50f-41a984d70701-istiod-ca-cert\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.855783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.855757 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-envoy\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.856473 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.856449 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4371e0f4-8593-40eb-a50f-41a984d70701-istio-podinfo\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.865313 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.865286 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-istio-token\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.865478 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.865457 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh7qc\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-kube-api-access-nh7qc\") pod \"e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:42.892309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:42.892280 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:43.029036 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:43.029001 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc"] Aug 03 16:40:43.030557 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:40:43.030533 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4371e0f4_8593_40eb_a50f_41a984d70701.slice/crio-cafd7fb6313f58c41e069cf28d381091e45ceba41490f7c9b74773bcd90201af WatchSource:0}: Error finding container cafd7fb6313f58c41e069cf28d381091e45ceba41490f7c9b74773bcd90201af: Status 404 returned error can't find the container with id cafd7fb6313f58c41e069cf28d381091e45ceba41490f7c9b74773bcd90201af Aug 03 16:40:43.033028 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:43.032993 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:40:43.033106 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:43.033056 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:40:43.033106 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:43.033094 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:40:43.807279 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:43.807244 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" event={"ID":"4371e0f4-8593-40eb-a50f-41a984d70701","Type":"ContainerStarted","Data":"23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378"} Aug 03 16:40:43.807691 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:43.807302 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" event={"ID":"4371e0f4-8593-40eb-a50f-41a984d70701","Type":"ContainerStarted","Data":"cafd7fb6313f58c41e069cf28d381091e45ceba41490f7c9b74773bcd90201af"} Aug 03 16:40:43.828957 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:43.828910 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" podStartSLOduration=1.828893538 podStartE2EDuration="1.828893538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:40:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:40:43.827670662 +0000 UTC m=+2182.385753247" watchObservedRunningTime="2026-08-03 16:40:43.828893538 +0000 UTC m=+2182.386976122" Aug 03 16:40:43.892543 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:43.892513 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:44.898261 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:44.898230 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:45.818425 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:45.818389 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:40:45.819386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:40:45.819366 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:41:00.820074 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:00.820037 2582 patch_prober.go:28] interesting pod/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Aug 03 16:41:00.820560 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:00.820106 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Aug 03 16:41:00.917036 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:00.917002 2582 generic.go:356] "Generic (PLEG): container finished" podID="4371e0f4-8593-40eb-a50f-41a984d70701" containerID="23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378" exitCode=0 Aug 03 16:41:00.917202 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:00.917076 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" event={"ID":"4371e0f4-8593-40eb-a50f-41a984d70701","Type":"ContainerDied","Data":"23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378"} Aug 03 16:41:00.917471 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:00.917444 2582 scope.go:117] "RemoveContainer" containerID="23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378" Aug 03 16:41:00.918483 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:00.918453 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:41:00.918569 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:00.918519 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:41:00.918569 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:00.918555 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:41:01.923803 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:01.923766 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" event={"ID":"4371e0f4-8593-40eb-a50f-41a984d70701","Type":"ContainerStarted","Data":"154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9"} Aug 03 16:41:02.892769 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:02.892734 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:41:02.893019 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:02.892997 2582 patch_prober.go:28] interesting pod/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Aug 03 16:41:02.893065 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:02.893043 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Aug 03 16:41:03.892961 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:03.892924 2582 patch_prober.go:28] interesting pod/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Aug 03 16:41:03.893371 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:03.892984 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Aug 03 16:41:04.893569 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:04.893532 2582 patch_prober.go:28] interesting pod/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Aug 03 16:41:04.893972 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:04.893599 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Aug 03 16:41:05.893438 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:05.893402 2582 patch_prober.go:28] interesting pod/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Aug 03 16:41:05.893640 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:05.893456 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Aug 03 16:41:06.896414 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:06.896382 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:41:06.896861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:06.896754 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:41:06.897369 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:41:06.897348 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:43:14.282320 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:14.282232 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc"] Aug 03 16:43:14.282843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:14.282442 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" containerID="cri-o://154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9" gracePeriod=30 Aug 03 16:43:19.526694 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.526670 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:43:19.595651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595543 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-certs\") pod \"4371e0f4-8593-40eb-a50f-41a984d70701\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " Aug 03 16:43:19.595651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595586 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4371e0f4-8593-40eb-a50f-41a984d70701-istiod-ca-cert\") pod \"4371e0f4-8593-40eb-a50f-41a984d70701\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " Aug 03 16:43:19.595875 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595649 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-credential-socket\") pod \"4371e0f4-8593-40eb-a50f-41a984d70701\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " Aug 03 16:43:19.595875 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595677 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-istio-token\") pod \"4371e0f4-8593-40eb-a50f-41a984d70701\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " Aug 03 16:43:19.595875 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595700 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-envoy\") pod \"4371e0f4-8593-40eb-a50f-41a984d70701\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " Aug 03 16:43:19.595875 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595752 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-socket\") pod \"4371e0f4-8593-40eb-a50f-41a984d70701\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " Aug 03 16:43:19.595875 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595799 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4371e0f4-8593-40eb-a50f-41a984d70701" (UID: "4371e0f4-8593-40eb-a50f-41a984d70701"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:43:19.595875 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595826 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-data\") pod \"4371e0f4-8593-40eb-a50f-41a984d70701\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " Aug 03 16:43:19.595875 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595862 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh7qc\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-kube-api-access-nh7qc\") pod \"4371e0f4-8593-40eb-a50f-41a984d70701\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " Aug 03 16:43:19.596223 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595894 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4371e0f4-8593-40eb-a50f-41a984d70701-istio-podinfo\") pod \"4371e0f4-8593-40eb-a50f-41a984d70701\" (UID: \"4371e0f4-8593-40eb-a50f-41a984d70701\") " Aug 03 16:43:19.596223 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.595988 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4371e0f4-8593-40eb-a50f-41a984d70701" (UID: "4371e0f4-8593-40eb-a50f-41a984d70701"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:43:19.596223 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.596079 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4371e0f4-8593-40eb-a50f-41a984d70701" (UID: "4371e0f4-8593-40eb-a50f-41a984d70701"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:43:19.596223 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.596018 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4371e0f4-8593-40eb-a50f-41a984d70701-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4371e0f4-8593-40eb-a50f-41a984d70701" (UID: "4371e0f4-8593-40eb-a50f-41a984d70701"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:43:19.596223 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.596130 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-data" (OuterVolumeSpecName: "istio-data") pod "4371e0f4-8593-40eb-a50f-41a984d70701" (UID: "4371e0f4-8593-40eb-a50f-41a984d70701"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:43:19.596421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.596249 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:43:19.596421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.596264 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:43:19.596421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.596278 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4371e0f4-8593-40eb-a50f-41a984d70701-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:43:19.596421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.596295 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:43:19.596421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.596313 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:43:19.597993 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.597963 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4371e0f4-8593-40eb-a50f-41a984d70701-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4371e0f4-8593-40eb-a50f-41a984d70701" (UID: "4371e0f4-8593-40eb-a50f-41a984d70701"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:43:19.598112 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.598043 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-istio-token" (OuterVolumeSpecName: "istio-token") pod "4371e0f4-8593-40eb-a50f-41a984d70701" (UID: "4371e0f4-8593-40eb-a50f-41a984d70701"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:43:19.598366 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.598339 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-kube-api-access-nh7qc" (OuterVolumeSpecName: "kube-api-access-nh7qc") pod "4371e0f4-8593-40eb-a50f-41a984d70701" (UID: "4371e0f4-8593-40eb-a50f-41a984d70701"). InnerVolumeSpecName "kube-api-access-nh7qc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:43:19.598366 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.598348 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4371e0f4-8593-40eb-a50f-41a984d70701" (UID: "4371e0f4-8593-40eb-a50f-41a984d70701"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:43:19.694164 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.694128 2582 generic.go:356] "Generic (PLEG): container finished" podID="4371e0f4-8593-40eb-a50f-41a984d70701" containerID="154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9" exitCode=0 Aug 03 16:43:19.694334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.694171 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" Aug 03 16:43:19.694334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.694215 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" event={"ID":"4371e0f4-8593-40eb-a50f-41a984d70701","Type":"ContainerDied","Data":"154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9"} Aug 03 16:43:19.694334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.694295 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc" event={"ID":"4371e0f4-8593-40eb-a50f-41a984d70701","Type":"ContainerDied","Data":"cafd7fb6313f58c41e069cf28d381091e45ceba41490f7c9b74773bcd90201af"} Aug 03 16:43:19.694334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.694317 2582 scope.go:117] "RemoveContainer" containerID="154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9" Aug 03 16:43:19.696904 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.696884 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:43:19.696904 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.696903 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4371e0f4-8593-40eb-a50f-41a984d70701-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:43:19.697031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.696914 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh7qc\" (UniqueName: \"kubernetes.io/projected/4371e0f4-8593-40eb-a50f-41a984d70701-kube-api-access-nh7qc\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:43:19.697031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.696923 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4371e0f4-8593-40eb-a50f-41a984d70701-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:43:19.703761 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.703742 2582 scope.go:117] "RemoveContainer" containerID="23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378" Aug 03 16:43:19.712151 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.712133 2582 scope.go:117] "RemoveContainer" containerID="154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9" Aug 03 16:43:19.712427 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:43:19.712406 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9\": container with ID starting with 154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9 not found: ID does not exist" containerID="154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9" Aug 03 16:43:19.712472 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.712446 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9"} err="failed to get container status \"154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9\": rpc error: code = NotFound desc = could not find container \"154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9\": container with ID starting with 154812f130891d856dee5bcc2a35b62de20b94a930ec9a35eefd9cce523a9eb9 not found: ID does not exist" Aug 03 16:43:19.712472 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.712465 2582 scope.go:117] "RemoveContainer" containerID="23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378" Aug 03 16:43:19.712743 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:43:19.712727 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378\": container with ID starting with 23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378 not found: ID does not exist" containerID="23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378" Aug 03 16:43:19.712814 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.712746 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378"} err="failed to get container status \"23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378\": rpc error: code = NotFound desc = could not find container \"23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378\": container with ID starting with 23f7f5947b609642d2ce0ed81165b2c8e54322e6f6e471235e54cb401d1ed378 not found: ID does not exist" Aug 03 16:43:19.718919 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.718900 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc"] Aug 03 16:43:19.720923 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.720904 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-65805bcc-openshift-default-677c85d9b-8zxsc"] Aug 03 16:43:19.996071 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:43:19.995994 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" path="/var/lib/kubelet/pods/4371e0f4-8593-40eb-a50f-41a984d70701/volumes" Aug 03 16:44:20.798605 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.798567 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72"] Aug 03 16:44:20.799078 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.799022 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" Aug 03 16:44:20.799078 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.799036 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" Aug 03 16:44:20.799078 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.799049 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" Aug 03 16:44:20.799078 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.799055 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" Aug 03 16:44:20.799216 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.799131 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" Aug 03 16:44:20.799216 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.799139 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4371e0f4-8593-40eb-a50f-41a984d70701" containerName="istio-proxy" Aug 03 16:44:20.802309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.802291 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:20.805270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.805252 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7ca182-openshift-default-dockercfg-tnmnj\"" Aug 03 16:44:20.816535 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.816511 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72"] Aug 03 16:44:20.957896 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.957863 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-socket\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:20.958107 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.957915 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed980145-073c-4721-8dad-31e9578b1d7b-istio-podinfo\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:20.958107 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.957945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-certs\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:20.958107 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.958025 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-data\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:20.958107 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.958068 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-istio-token\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:20.958107 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.958099 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed980145-073c-4721-8dad-31e9578b1d7b-istiod-ca-cert\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:20.958350 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.958151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-credential-socket\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:20.958350 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.958177 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-envoy\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:20.958350 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:20.958226 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgsr8\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-kube-api-access-fgsr8\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.058783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.058698 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-credential-socket\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.058783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.058734 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-envoy\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.058783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.058766 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgsr8\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-kube-api-access-fgsr8\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059059 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.058882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-socket\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059059 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059025 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed980145-073c-4721-8dad-31e9578b1d7b-istio-podinfo\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059174 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059061 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-certs\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059174 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059089 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-credential-socket\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059174 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059128 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-data\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059174 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059161 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-istio-token\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059205 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed980145-073c-4721-8dad-31e9578b1d7b-istiod-ca-cert\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059280 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-socket\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059353 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-certs\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059522 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059442 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-data\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.059940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.059917 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed980145-073c-4721-8dad-31e9578b1d7b-istiod-ca-cert\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.061184 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.061165 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-envoy\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.061487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.061468 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed980145-073c-4721-8dad-31e9578b1d7b-istio-podinfo\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.067651 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.067608 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-istio-token\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.072368 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.072345 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgsr8\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-kube-api-access-fgsr8\") pod \"e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.112240 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.112205 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:21.249938 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.249907 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72"] Aug 03 16:44:21.251252 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:44:21.251226 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded980145_073c_4721_8dad_31e9578b1d7b.slice/crio-f3f2a40a72c14c82d9338b2ce2e99ff9a247a3f65344a41c8bf420c4248dd1eb WatchSource:0}: Error finding container f3f2a40a72c14c82d9338b2ce2e99ff9a247a3f65344a41c8bf420c4248dd1eb: Status 404 returned error can't find the container with id f3f2a40a72c14c82d9338b2ce2e99ff9a247a3f65344a41c8bf420c4248dd1eb Aug 03 16:44:21.253419 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.253386 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:44:21.253502 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.253466 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:44:21.253543 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:21.253517 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:44:22.038502 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:22.038464 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" event={"ID":"ed980145-073c-4721-8dad-31e9578b1d7b","Type":"ContainerStarted","Data":"56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90"} Aug 03 16:44:22.038922 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:22.038523 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" event={"ID":"ed980145-073c-4721-8dad-31e9578b1d7b","Type":"ContainerStarted","Data":"f3f2a40a72c14c82d9338b2ce2e99ff9a247a3f65344a41c8bf420c4248dd1eb"} Aug 03 16:44:22.075579 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:22.075531 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" podStartSLOduration=2.075517885 podStartE2EDuration="2.075517885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:44:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:44:22.073576772 +0000 UTC m=+2400.631659356" watchObservedRunningTime="2026-08-03 16:44:22.075517885 +0000 UTC m=+2400.633600469" Aug 03 16:44:22.112940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:22.112909 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:22.117593 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:22.117566 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:23.044094 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:23.044054 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:23.045241 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:23.045222 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:38.045031 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:38.044999 2582 patch_prober.go:28] interesting pod/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 03 16:44:38.045516 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:38.045067 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 03 16:44:38.126339 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:38.126308 2582 generic.go:356] "Generic (PLEG): container finished" podID="ed980145-073c-4721-8dad-31e9578b1d7b" containerID="56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90" exitCode=0 Aug 03 16:44:38.126507 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:38.126363 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" event={"ID":"ed980145-073c-4721-8dad-31e9578b1d7b","Type":"ContainerDied","Data":"56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90"} Aug 03 16:44:38.126730 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:38.126717 2582 scope.go:117] "RemoveContainer" containerID="56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90" Aug 03 16:44:38.127752 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:38.127725 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:44:38.127859 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:38.127804 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:44:38.127859 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:38.127849 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:44:39.135004 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:39.134960 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" event={"ID":"ed980145-073c-4721-8dad-31e9578b1d7b","Type":"ContainerStarted","Data":"0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64"} Aug 03 16:44:40.112860 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:40.112825 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:40.113046 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:40.113021 2582 patch_prober.go:28] interesting pod/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 03 16:44:40.113094 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:40.113065 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 03 16:44:41.112963 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:41.112926 2582 patch_prober.go:28] interesting pod/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 03 16:44:41.113357 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:41.112981 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 03 16:44:42.112910 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:42.112872 2582 patch_prober.go:28] interesting pod/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 03 16:44:42.113118 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:42.112942 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 03 16:44:43.112556 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:43.112518 2582 patch_prober.go:28] interesting pod/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Aug 03 16:44:43.112767 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:43.112582 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Aug 03 16:44:44.116407 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:44.116369 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:44.116894 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:44.116665 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:44.117202 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:44.117185 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:44:49.673532 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.673493 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf"] Aug 03 16:44:49.676271 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.676248 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.679096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.679070 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d7cee2-openshift-default-dockercfg-8lc7r\"" Aug 03 16:44:49.688795 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.688770 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf"] Aug 03 16:44:49.748745 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.748714 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec21b454-4faf-4881-a652-f07efb11e526-istiod-ca-cert\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.748931 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.748780 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-data\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.748931 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.748860 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec21b454-4faf-4881-a652-f07efb11e526-istio-podinfo\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.748931 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.748925 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrbk6\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-kube-api-access-nrbk6\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.749103 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.748948 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-certs\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.749103 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.748997 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-envoy\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.749103 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.749040 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-istio-token\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.749103 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.749081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-socket\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.749256 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.749108 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-credential-socket\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.849980 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.849940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrbk6\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-kube-api-access-nrbk6\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.849980 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.849979 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-certs\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850000 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-envoy\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850023 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-istio-token\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850146 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-socket\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850190 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-credential-socket\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850247 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec21b454-4faf-4881-a652-f07efb11e526-istiod-ca-cert\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850309 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-data\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850363 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec21b454-4faf-4881-a652-f07efb11e526-istio-podinfo\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850416 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-certs\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850488 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-socket\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850726 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850586 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-credential-socket\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850726 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850673 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-data\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.850965 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.850946 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec21b454-4faf-4881-a652-f07efb11e526-istiod-ca-cert\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.852630 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.852596 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-envoy\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.852779 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.852763 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec21b454-4faf-4881-a652-f07efb11e526-istio-podinfo\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.858642 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.858604 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-istio-token\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.859166 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.859150 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrbk6\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-kube-api-access-nrbk6\") pod \"e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:49.986422 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:49.986330 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:50.134184 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:44:50.134154 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec21b454_4faf_4881_a652_f07efb11e526.slice/crio-d57df87f883e0667da9771383dfaca9c4305f646f076b2570d61297e061fcae7 WatchSource:0}: Error finding container d57df87f883e0667da9771383dfaca9c4305f646f076b2570d61297e061fcae7: Status 404 returned error can't find the container with id d57df87f883e0667da9771383dfaca9c4305f646f076b2570d61297e061fcae7 Aug 03 16:44:50.134387 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:50.134363 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf"] Aug 03 16:44:50.136527 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:50.136497 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:44:50.136644 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:50.136561 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:44:50.136644 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:50.136586 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:44:50.212294 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:50.212261 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" event={"ID":"ec21b454-4faf-4881-a652-f07efb11e526","Type":"ContainerStarted","Data":"24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449"} Aug 03 16:44:50.212458 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:50.212326 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" event={"ID":"ec21b454-4faf-4881-a652-f07efb11e526","Type":"ContainerStarted","Data":"d57df87f883e0667da9771383dfaca9c4305f646f076b2570d61297e061fcae7"} Aug 03 16:44:50.247607 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:50.247375 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" podStartSLOduration=1.247353212 podStartE2EDuration="1.247353212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:44:50.247121628 +0000 UTC m=+2428.805204214" watchObservedRunningTime="2026-08-03 16:44:50.247353212 +0000 UTC m=+2428.805435798" Aug 03 16:44:50.987003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:50.986965 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:51.994943 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:51.994913 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:52.228191 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:52.228153 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:44:52.231145 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:44:52.231119 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:45:05.312398 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:05.312363 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec21b454-4faf-4881-a652-f07efb11e526" containerID="24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449" exitCode=0 Aug 03 16:45:05.313006 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:05.312442 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" event={"ID":"ec21b454-4faf-4881-a652-f07efb11e526","Type":"ContainerDied","Data":"24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449"} Aug 03 16:45:05.313006 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:05.312982 2582 scope.go:117] "RemoveContainer" containerID="24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449" Aug 03 16:45:05.314039 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:05.314015 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:45:05.314094 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:05.314080 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:45:05.314130 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:05.314111 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:45:06.319892 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:06.319861 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" event={"ID":"ec21b454-4faf-4881-a652-f07efb11e526","Type":"ContainerStarted","Data":"1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038"} Aug 03 16:45:06.986519 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:06.986480 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:45:06.987699 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:06.987673 2582 patch_prober.go:28] interesting pod/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Aug 03 16:45:06.987770 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:06.987739 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Aug 03 16:45:07.228547 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:07.228513 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:45:07.986779 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:07.986743 2582 patch_prober.go:28] interesting pod/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Aug 03 16:45:07.987170 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:07.986827 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Aug 03 16:45:08.986826 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:08.986786 2582 patch_prober.go:28] interesting pod/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Aug 03 16:45:08.987233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:08.986845 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Aug 03 16:45:09.987482 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:09.987449 2582 patch_prober.go:28] interesting pod/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Aug 03 16:45:09.987869 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:09.987505 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Aug 03 16:45:10.987123 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:10.987081 2582 patch_prober.go:28] interesting pod/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Aug 03 16:45:10.987330 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:10.987142 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Aug 03 16:45:11.995676 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:11.995650 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:45:11.996593 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:45:11.996575 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:47:11.115631 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:11.115589 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72"] Aug 03 16:47:11.116129 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:11.115808 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" containerID="cri-o://0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64" gracePeriod=30 Aug 03 16:47:14.119259 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:14.119221 2582 patch_prober.go:28] interesting pod/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 03 16:47:14.119674 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:14.119295 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 03 16:47:16.365949 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.365926 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:47:16.463578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463490 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-istio-token\") pod \"ed980145-073c-4721-8dad-31e9578b1d7b\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " Aug 03 16:47:16.463578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463525 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-data\") pod \"ed980145-073c-4721-8dad-31e9578b1d7b\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " Aug 03 16:47:16.463578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463556 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-envoy\") pod \"ed980145-073c-4721-8dad-31e9578b1d7b\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " Aug 03 16:47:16.463578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463572 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-credential-socket\") pod \"ed980145-073c-4721-8dad-31e9578b1d7b\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " Aug 03 16:47:16.463940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463602 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-certs\") pod \"ed980145-073c-4721-8dad-31e9578b1d7b\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " Aug 03 16:47:16.463940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463744 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgsr8\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-kube-api-access-fgsr8\") pod \"ed980145-073c-4721-8dad-31e9578b1d7b\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " Aug 03 16:47:16.463940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463826 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed980145-073c-4721-8dad-31e9578b1d7b-istio-podinfo\") pod \"ed980145-073c-4721-8dad-31e9578b1d7b\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " Aug 03 16:47:16.463940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463852 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-data" (OuterVolumeSpecName: "istio-data") pod "ed980145-073c-4721-8dad-31e9578b1d7b" (UID: "ed980145-073c-4721-8dad-31e9578b1d7b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:16.463940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463884 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-socket\") pod \"ed980145-073c-4721-8dad-31e9578b1d7b\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " Aug 03 16:47:16.463940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463881 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ed980145-073c-4721-8dad-31e9578b1d7b" (UID: "ed980145-073c-4721-8dad-31e9578b1d7b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:16.463940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463937 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ed980145-073c-4721-8dad-31e9578b1d7b" (UID: "ed980145-073c-4721-8dad-31e9578b1d7b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:16.464308 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.463943 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed980145-073c-4721-8dad-31e9578b1d7b-istiod-ca-cert\") pod \"ed980145-073c-4721-8dad-31e9578b1d7b\" (UID: \"ed980145-073c-4721-8dad-31e9578b1d7b\") " Aug 03 16:47:16.464308 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.464108 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ed980145-073c-4721-8dad-31e9578b1d7b" (UID: "ed980145-073c-4721-8dad-31e9578b1d7b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:16.464421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.464308 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:16.464421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.464329 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:16.464421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.464332 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed980145-073c-4721-8dad-31e9578b1d7b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ed980145-073c-4721-8dad-31e9578b1d7b" (UID: "ed980145-073c-4721-8dad-31e9578b1d7b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:47:16.464421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.464344 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:16.464421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.464378 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:16.465941 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.465909 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ed980145-073c-4721-8dad-31e9578b1d7b" (UID: "ed980145-073c-4721-8dad-31e9578b1d7b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:16.466061 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.465963 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-kube-api-access-fgsr8" (OuterVolumeSpecName: "kube-api-access-fgsr8") pod "ed980145-073c-4721-8dad-31e9578b1d7b" (UID: "ed980145-073c-4721-8dad-31e9578b1d7b"). InnerVolumeSpecName "kube-api-access-fgsr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:47:16.466061 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.465993 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-istio-token" (OuterVolumeSpecName: "istio-token") pod "ed980145-073c-4721-8dad-31e9578b1d7b" (UID: "ed980145-073c-4721-8dad-31e9578b1d7b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:47:16.466061 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.466052 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed980145-073c-4721-8dad-31e9578b1d7b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ed980145-073c-4721-8dad-31e9578b1d7b" (UID: "ed980145-073c-4721-8dad-31e9578b1d7b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:47:16.565317 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.565276 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgsr8\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-kube-api-access-fgsr8\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:16.565317 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.565310 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed980145-073c-4721-8dad-31e9578b1d7b-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:16.565317 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.565322 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed980145-073c-4721-8dad-31e9578b1d7b-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:16.565558 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.565333 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed980145-073c-4721-8dad-31e9578b1d7b-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:16.565558 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:16.565344 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed980145-073c-4721-8dad-31e9578b1d7b-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:17.047003 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.046967 2582 generic.go:356] "Generic (PLEG): container finished" podID="ed980145-073c-4721-8dad-31e9578b1d7b" containerID="0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64" exitCode=0 Aug 03 16:47:17.047178 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.047010 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" Aug 03 16:47:17.047178 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.047049 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" event={"ID":"ed980145-073c-4721-8dad-31e9578b1d7b","Type":"ContainerDied","Data":"0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64"} Aug 03 16:47:17.047178 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.047129 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72" event={"ID":"ed980145-073c-4721-8dad-31e9578b1d7b","Type":"ContainerDied","Data":"f3f2a40a72c14c82d9338b2ce2e99ff9a247a3f65344a41c8bf420c4248dd1eb"} Aug 03 16:47:17.047178 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.047153 2582 scope.go:117] "RemoveContainer" containerID="0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64" Aug 03 16:47:17.056605 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.056587 2582 scope.go:117] "RemoveContainer" containerID="56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90" Aug 03 16:47:17.064990 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.064974 2582 scope.go:117] "RemoveContainer" containerID="0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64" Aug 03 16:47:17.065242 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:47:17.065225 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64\": container with ID starting with 0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64 not found: ID does not exist" containerID="0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64" Aug 03 16:47:17.065292 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.065251 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64"} err="failed to get container status \"0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64\": rpc error: code = NotFound desc = could not find container \"0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64\": container with ID starting with 0d14c83acbeea94ac21ebc1572af421b1fea3362dcdee33875ca12758ef2bc64 not found: ID does not exist" Aug 03 16:47:17.065292 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.065265 2582 scope.go:117] "RemoveContainer" containerID="56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90" Aug 03 16:47:17.065493 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:47:17.065475 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90\": container with ID starting with 56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90 not found: ID does not exist" containerID="56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90" Aug 03 16:47:17.065550 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.065504 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90"} err="failed to get container status \"56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90\": rpc error: code = NotFound desc = could not find container \"56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90\": container with ID starting with 56ef8f525e7c2882487b92fa6e78b00493219f467ba69e20a2f6b34bb2dd8b90 not found: ID does not exist" Aug 03 16:47:17.071297 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.071271 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72"] Aug 03 16:47:17.075848 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.075825 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7ca182-openshift-default-6c4555b6f4-hzl72"] Aug 03 16:47:17.996166 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:17.996133 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" path="/var/lib/kubelet/pods/ed980145-073c-4721-8dad-31e9578b1d7b/volumes" Aug 03 16:47:29.090914 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.090882 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf"] Aug 03 16:47:29.091347 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.091096 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" containerID="cri-o://1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038" gracePeriod=30 Aug 03 16:47:29.958866 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.958824 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq"] Aug 03 16:47:29.959271 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.959256 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" Aug 03 16:47:29.959271 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.959270 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" Aug 03 16:47:29.959422 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.959280 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" Aug 03 16:47:29.959422 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.959286 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" Aug 03 16:47:29.959422 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.959357 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" Aug 03 16:47:29.959422 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.959366 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed980145-073c-4721-8dad-31e9578b1d7b" containerName="istio-proxy" Aug 03 16:47:29.961430 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.961384 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:29.964499 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.964481 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-deec1a-openshift-default-dockercfg-9wjz2\"" Aug 03 16:47:29.976835 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.976810 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq"] Aug 03 16:47:29.986910 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.986882 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-podinfo\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:29.987038 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.986918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-credential-socket\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:29.987038 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.986945 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-data\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:29.987038 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.987013 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-token\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:29.987221 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.987051 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istiod-ca-cert\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:29.987221 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.987071 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-certs\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:29.987221 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.987101 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-envoy\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:29.987221 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.987125 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-socket\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:29.987221 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:29.987149 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkn7s\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-kube-api-access-pkn7s\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.087809 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.087777 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-token\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088015 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.087817 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istiod-ca-cert\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088015 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.087840 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-certs\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088015 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.087904 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-envoy\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088015 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.087934 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-socket\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088015 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.087967 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkn7s\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-kube-api-access-pkn7s\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088279 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.088026 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-podinfo\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088279 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.088055 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-credential-socket\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088279 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.088084 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-data\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088430 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.088354 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-certs\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.088420 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-socket\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088487 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.088460 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-data\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088711 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.088693 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istiod-ca-cert\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.088790 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.088714 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-credential-socket\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.090772 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.090746 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-envoy\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.090922 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.090908 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-podinfo\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.100913 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.100881 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-token\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.101214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.101052 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkn7s\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-kube-api-access-pkn7s\") pod \"e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.272055 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.272025 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:30.407739 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.407713 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq"] Aug 03 16:47:30.409587 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:47:30.409559 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56aefbfd_00cb_4dee_a618_38b4f5ff60ef.slice/crio-6e134760935a5978cf9193a1983fcf33c47272827dcb0db0112ab3ccef391dd9 WatchSource:0}: Error finding container 6e134760935a5978cf9193a1983fcf33c47272827dcb0db0112ab3ccef391dd9: Status 404 returned error can't find the container with id 6e134760935a5978cf9193a1983fcf33c47272827dcb0db0112ab3ccef391dd9 Aug 03 16:47:30.411681 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.411650 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:47:30.411771 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.411714 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:47:30.411771 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:30.411742 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:47:31.125561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:31.125526 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" event={"ID":"56aefbfd-00cb-4dee-a618-38b4f5ff60ef","Type":"ContainerStarted","Data":"22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad"} Aug 03 16:47:31.125972 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:31.125578 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" event={"ID":"56aefbfd-00cb-4dee-a618-38b4f5ff60ef","Type":"ContainerStarted","Data":"6e134760935a5978cf9193a1983fcf33c47272827dcb0db0112ab3ccef391dd9"} Aug 03 16:47:31.148768 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:31.148716 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" podStartSLOduration=2.148701597 podStartE2EDuration="2.148701597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:47:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:47:31.147556837 +0000 UTC m=+2589.705639424" watchObservedRunningTime="2026-08-03 16:47:31.148701597 +0000 UTC m=+2589.706784181" Aug 03 16:47:31.272870 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:31.272836 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:31.277264 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:31.277240 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:32.130930 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:32.130899 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:32.131926 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:32.131904 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:34.345939 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.345917 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:47:34.422981 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.422897 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-socket\") pod \"ec21b454-4faf-4881-a652-f07efb11e526\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " Aug 03 16:47:34.422981 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.422952 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec21b454-4faf-4881-a652-f07efb11e526-istiod-ca-cert\") pod \"ec21b454-4faf-4881-a652-f07efb11e526\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " Aug 03 16:47:34.423250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.422982 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-envoy\") pod \"ec21b454-4faf-4881-a652-f07efb11e526\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " Aug 03 16:47:34.423250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423016 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-istio-token\") pod \"ec21b454-4faf-4881-a652-f07efb11e526\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " Aug 03 16:47:34.423250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423061 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec21b454-4faf-4881-a652-f07efb11e526-istio-podinfo\") pod \"ec21b454-4faf-4881-a652-f07efb11e526\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " Aug 03 16:47:34.423250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423105 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-certs\") pod \"ec21b454-4faf-4881-a652-f07efb11e526\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " Aug 03 16:47:34.423250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423129 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrbk6\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-kube-api-access-nrbk6\") pod \"ec21b454-4faf-4881-a652-f07efb11e526\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " Aug 03 16:47:34.423250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423156 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-credential-socket\") pod \"ec21b454-4faf-4881-a652-f07efb11e526\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " Aug 03 16:47:34.423250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423191 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-data\") pod \"ec21b454-4faf-4881-a652-f07efb11e526\" (UID: \"ec21b454-4faf-4881-a652-f07efb11e526\") " Aug 03 16:47:34.423250 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423233 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec21b454-4faf-4881-a652-f07efb11e526" (UID: "ec21b454-4faf-4881-a652-f07efb11e526"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:34.423657 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423486 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:34.423657 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423489 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec21b454-4faf-4881-a652-f07efb11e526-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec21b454-4faf-4881-a652-f07efb11e526" (UID: "ec21b454-4faf-4881-a652-f07efb11e526"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:47:34.423657 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423590 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec21b454-4faf-4881-a652-f07efb11e526" (UID: "ec21b454-4faf-4881-a652-f07efb11e526"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:34.423821 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423695 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec21b454-4faf-4881-a652-f07efb11e526" (UID: "ec21b454-4faf-4881-a652-f07efb11e526"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:34.423821 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.423608 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec21b454-4faf-4881-a652-f07efb11e526" (UID: "ec21b454-4faf-4881-a652-f07efb11e526"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:34.425209 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.425191 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec21b454-4faf-4881-a652-f07efb11e526" (UID: "ec21b454-4faf-4881-a652-f07efb11e526"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:47:34.425363 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.425344 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec21b454-4faf-4881-a652-f07efb11e526-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec21b454-4faf-4881-a652-f07efb11e526" (UID: "ec21b454-4faf-4881-a652-f07efb11e526"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:47:34.425571 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.425550 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec21b454-4faf-4881-a652-f07efb11e526" (UID: "ec21b454-4faf-4881-a652-f07efb11e526"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:47:34.425694 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.425565 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-kube-api-access-nrbk6" (OuterVolumeSpecName: "kube-api-access-nrbk6") pod "ec21b454-4faf-4881-a652-f07efb11e526" (UID: "ec21b454-4faf-4881-a652-f07efb11e526"). InnerVolumeSpecName "kube-api-access-nrbk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:47:34.524119 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.524087 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:34.524119 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.524114 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:34.524119 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.524122 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec21b454-4faf-4881-a652-f07efb11e526-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:34.524352 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.524131 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:34.524352 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.524141 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrbk6\" (UniqueName: \"kubernetes.io/projected/ec21b454-4faf-4881-a652-f07efb11e526-kube-api-access-nrbk6\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:34.524352 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.524152 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:34.524352 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.524160 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec21b454-4faf-4881-a652-f07efb11e526-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:34.524352 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:34.524168 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec21b454-4faf-4881-a652-f07efb11e526-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:47:35.158068 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.158037 2582 generic.go:356] "Generic (PLEG): container finished" podID="ec21b454-4faf-4881-a652-f07efb11e526" containerID="1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038" exitCode=0 Aug 03 16:47:35.158257 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.158081 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" Aug 03 16:47:35.158257 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.158118 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" event={"ID":"ec21b454-4faf-4881-a652-f07efb11e526","Type":"ContainerDied","Data":"1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038"} Aug 03 16:47:35.158257 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.158171 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf" event={"ID":"ec21b454-4faf-4881-a652-f07efb11e526","Type":"ContainerDied","Data":"d57df87f883e0667da9771383dfaca9c4305f646f076b2570d61297e061fcae7"} Aug 03 16:47:35.158257 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.158192 2582 scope.go:117] "RemoveContainer" containerID="1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038" Aug 03 16:47:35.167602 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.167583 2582 scope.go:117] "RemoveContainer" containerID="24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449" Aug 03 16:47:35.175437 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.175421 2582 scope.go:117] "RemoveContainer" containerID="1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038" Aug 03 16:47:35.175704 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:47:35.175685 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038\": container with ID starting with 1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038 not found: ID does not exist" containerID="1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038" Aug 03 16:47:35.175773 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.175711 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038"} err="failed to get container status \"1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038\": rpc error: code = NotFound desc = could not find container \"1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038\": container with ID starting with 1cb4c7af94cc70f90393f5a88611577f0f7bea11c955737924da591590d96038 not found: ID does not exist" Aug 03 16:47:35.175773 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.175729 2582 scope.go:117] "RemoveContainer" containerID="24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449" Aug 03 16:47:35.175954 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:47:35.175937 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449\": container with ID starting with 24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449 not found: ID does not exist" containerID="24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449" Aug 03 16:47:35.175992 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.175960 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449"} err="failed to get container status \"24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449\": rpc error: code = NotFound desc = could not find container \"24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449\": container with ID starting with 24b19dc8ef3da0feb3cc824579960580d8c08ff96cec03c60bae2759904cd449 not found: ID does not exist" Aug 03 16:47:35.181714 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.181693 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf"] Aug 03 16:47:35.187966 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.187946 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d7cee2-openshift-default-7988d98676-qn5gf"] Aug 03 16:47:35.995518 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:35.995487 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec21b454-4faf-4881-a652-f07efb11e526" path="/var/lib/kubelet/pods/ec21b454-4faf-4881-a652-f07efb11e526/volumes" Aug 03 16:47:46.219137 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:46.219050 2582 generic.go:356] "Generic (PLEG): container finished" podID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerID="22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad" exitCode=0 Aug 03 16:47:46.219666 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:46.219129 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" event={"ID":"56aefbfd-00cb-4dee-a618-38b4f5ff60ef","Type":"ContainerDied","Data":"22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad"} Aug 03 16:47:46.219745 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:46.219682 2582 scope.go:117] "RemoveContainer" containerID="22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad" Aug 03 16:47:46.220711 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:46.220681 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:47:46.220818 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:46.220745 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:47:46.220818 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:46.220777 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:47:47.131222 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:47.131187 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:47.225999 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:47.225969 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" event={"ID":"56aefbfd-00cb-4dee-a618-38b4f5ff60ef","Type":"ContainerStarted","Data":"2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966"} Aug 03 16:47:47.272638 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:47.272591 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:47.273737 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:47.273712 2582 patch_prober.go:28] interesting pod/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Aug 03 16:47:47.273856 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:47.273759 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Aug 03 16:47:48.273248 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:48.273203 2582 patch_prober.go:28] interesting pod/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Aug 03 16:47:48.273760 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:48.273278 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Aug 03 16:47:49.272406 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:49.272371 2582 patch_prober.go:28] interesting pod/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Aug 03 16:47:49.272585 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:49.272433 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Aug 03 16:47:50.273234 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:50.273196 2582 patch_prober.go:28] interesting pod/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Aug 03 16:47:50.273748 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:50.273257 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Aug 03 16:47:51.273355 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:51.273316 2582 patch_prober.go:28] interesting pod/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Aug 03 16:47:51.273859 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:51.273387 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Aug 03 16:47:52.276493 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:52.276464 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:52.276918 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:52.276737 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:52.277345 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:52.277326 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:47:58.876361 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.876332 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78"] Aug 03 16:47:58.876783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.876747 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" Aug 03 16:47:58.876783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.876759 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" Aug 03 16:47:58.876783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.876773 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" Aug 03 16:47:58.876783 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.876778 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" Aug 03 16:47:58.876971 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.876839 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" Aug 03 16:47:58.876971 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.876846 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec21b454-4faf-4881-a652-f07efb11e526" containerName="istio-proxy" Aug 03 16:47:58.880061 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.880044 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:58.882674 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.882648 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9ffb05-openshift-default-dockercfg-jwj7g\"" Aug 03 16:47:58.894190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.894167 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78"] Aug 03 16:47:58.956060 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.956024 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-podinfo\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:58.956230 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.956080 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-certs\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:58.956230 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.956163 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-envoy\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:58.956230 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.956194 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7st8\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-kube-api-access-k7st8\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:58.956339 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.956288 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-token\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:58.956339 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.956318 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11fcf8c-7b30-4a21-9144-44285019e5a1-istiod-ca-cert\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:58.956400 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.956351 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-socket\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:58.956400 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.956367 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-credential-socket\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:58.956400 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:58.956389 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-data\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.057841 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.057807 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-token\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058047 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.057854 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11fcf8c-7b30-4a21-9144-44285019e5a1-istiod-ca-cert\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058047 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.057940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-socket\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058047 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.057965 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-credential-socket\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058047 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.057989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-data\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058274 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.058069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-podinfo\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058274 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.058177 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-certs\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058274 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.058238 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-envoy\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058274 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.058266 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7st8\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-kube-api-access-k7st8\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058499 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.058345 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-credential-socket\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058781 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.058733 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11fcf8c-7b30-4a21-9144-44285019e5a1-istiod-ca-cert\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058920 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.058896 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-certs\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.058995 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.058963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-socket\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.059048 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.058989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-data\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.060486 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.060467 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-envoy\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.060732 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.060713 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-podinfo\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.067012 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.066989 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-token\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.067343 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.067321 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7st8\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-kube-api-access-k7st8\") pod \"e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.190585 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.190498 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:47:59.328360 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.328333 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78"] Aug 03 16:47:59.330016 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:47:59.329991 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf11fcf8c_7b30_4a21_9144_44285019e5a1.slice/crio-8c8ff496fbc871375a9af3eb9a6897f9feb61c0c483523ee7dd684a1f03331c4 WatchSource:0}: Error finding container 8c8ff496fbc871375a9af3eb9a6897f9feb61c0c483523ee7dd684a1f03331c4: Status 404 returned error can't find the container with id 8c8ff496fbc871375a9af3eb9a6897f9feb61c0c483523ee7dd684a1f03331c4 Aug 03 16:47:59.339191 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.339150 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:47:59.339281 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.339239 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:47:59.339326 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:47:59.339283 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:48:00.315910 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:00.315880 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" event={"ID":"f11fcf8c-7b30-4a21-9144-44285019e5a1","Type":"ContainerStarted","Data":"28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457"} Aug 03 16:48:00.316290 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:00.315927 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" event={"ID":"f11fcf8c-7b30-4a21-9144-44285019e5a1","Type":"ContainerStarted","Data":"8c8ff496fbc871375a9af3eb9a6897f9feb61c0c483523ee7dd684a1f03331c4"} Aug 03 16:48:00.340183 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:00.340128 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" podStartSLOduration=2.340112468 podStartE2EDuration="2.340112468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:47:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:48:00.337442727 +0000 UTC m=+2618.895525345" watchObservedRunningTime="2026-08-03 16:48:00.340112468 +0000 UTC m=+2618.898195052" Aug 03 16:48:01.191278 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:01.191237 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:48:01.196171 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:01.196147 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:48:01.322022 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:01.321987 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:48:01.322924 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:01.322903 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:48:15.406938 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:15.406904 2582 generic.go:356] "Generic (PLEG): container finished" podID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerID="28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457" exitCode=0 Aug 03 16:48:15.407351 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:15.406974 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" event={"ID":"f11fcf8c-7b30-4a21-9144-44285019e5a1","Type":"ContainerDied","Data":"28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457"} Aug 03 16:48:15.407399 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:15.407354 2582 scope.go:117] "RemoveContainer" containerID="28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457" Aug 03 16:48:15.408423 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:15.408396 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:48:15.408492 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:15.408476 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:48:15.408530 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:15.408517 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:48:16.322566 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:16.322533 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:48:16.413817 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:16.413770 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" event={"ID":"f11fcf8c-7b30-4a21-9144-44285019e5a1","Type":"ContainerStarted","Data":"775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b"} Aug 03 16:48:17.190991 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:17.190958 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:48:17.192047 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:17.192023 2582 patch_prober.go:28] interesting pod/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 03 16:48:17.192171 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:17.192075 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 03 16:48:18.191207 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:18.191169 2582 patch_prober.go:28] interesting pod/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 03 16:48:18.191588 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:18.191238 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 03 16:48:19.191857 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:19.191817 2582 patch_prober.go:28] interesting pod/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 03 16:48:19.192320 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:19.191887 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 03 16:48:20.191338 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:20.191303 2582 patch_prober.go:28] interesting pod/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 03 16:48:20.191515 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:20.191368 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 03 16:48:21.191285 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:21.191247 2582 patch_prober.go:28] interesting pod/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Aug 03 16:48:21.191679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:21.191310 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Aug 03 16:48:22.195325 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:22.195295 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:48:22.195707 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:22.195547 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:48:22.196286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:22.196270 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:48:49.036581 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:49.036549 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq"] Aug 03 16:48:49.037186 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:49.036768 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" containerID="cri-o://2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966" gracePeriod=30 Aug 03 16:48:52.279459 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:52.279424 2582 patch_prober.go:28] interesting pod/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 03 16:48:52.279925 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:52.279489 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 03 16:48:54.283190 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.283166 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:48:54.391085 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.390998 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkn7s\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-kube-api-access-pkn7s\") pod \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " Aug 03 16:48:54.391085 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391066 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istiod-ca-cert\") pod \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " Aug 03 16:48:54.391315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391100 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-certs\") pod \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " Aug 03 16:48:54.391315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391126 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-socket\") pod \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " Aug 03 16:48:54.391315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391158 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-podinfo\") pod \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " Aug 03 16:48:54.391315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391181 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-token\") pod \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " Aug 03 16:48:54.391315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391217 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-data\") pod \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " Aug 03 16:48:54.391315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391255 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-credential-socket\") pod \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " Aug 03 16:48:54.391315 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391304 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-envoy\") pod \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\" (UID: \"56aefbfd-00cb-4dee-a618-38b4f5ff60ef\") " Aug 03 16:48:54.391706 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391456 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56aefbfd-00cb-4dee-a618-38b4f5ff60ef" (UID: "56aefbfd-00cb-4dee-a618-38b4f5ff60ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:48:54.391706 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391548 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56aefbfd-00cb-4dee-a618-38b4f5ff60ef" (UID: "56aefbfd-00cb-4dee-a618-38b4f5ff60ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:48:54.391706 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391450 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56aefbfd-00cb-4dee-a618-38b4f5ff60ef" (UID: "56aefbfd-00cb-4dee-a618-38b4f5ff60ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:48:54.391706 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391686 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "56aefbfd-00cb-4dee-a618-38b4f5ff60ef" (UID: "56aefbfd-00cb-4dee-a618-38b4f5ff60ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:48:54.391706 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391702 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:48:54.391972 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391721 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:48:54.391972 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.391822 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56aefbfd-00cb-4dee-a618-38b4f5ff60ef" (UID: "56aefbfd-00cb-4dee-a618-38b4f5ff60ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:48:54.393309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.393280 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-kube-api-access-pkn7s" (OuterVolumeSpecName: "kube-api-access-pkn7s") pod "56aefbfd-00cb-4dee-a618-38b4f5ff60ef" (UID: "56aefbfd-00cb-4dee-a618-38b4f5ff60ef"). InnerVolumeSpecName "kube-api-access-pkn7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:48:54.393309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.393285 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56aefbfd-00cb-4dee-a618-38b4f5ff60ef" (UID: "56aefbfd-00cb-4dee-a618-38b4f5ff60ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:48:54.393471 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.393398 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56aefbfd-00cb-4dee-a618-38b4f5ff60ef" (UID: "56aefbfd-00cb-4dee-a618-38b4f5ff60ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:48:54.393471 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.393435 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "56aefbfd-00cb-4dee-a618-38b4f5ff60ef" (UID: "56aefbfd-00cb-4dee-a618-38b4f5ff60ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:48:54.492413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.492379 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:48:54.492413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.492407 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:48:54.492413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.492418 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:48:54.492667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.492429 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:48:54.492667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.492441 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:48:54.492667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.492452 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:48:54.492667 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.492463 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkn7s\" (UniqueName: \"kubernetes.io/projected/56aefbfd-00cb-4dee-a618-38b4f5ff60ef-kube-api-access-pkn7s\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:48:54.631114 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.631083 2582 generic.go:356] "Generic (PLEG): container finished" podID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerID="2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966" exitCode=0 Aug 03 16:48:54.631286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.631128 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" Aug 03 16:48:54.631286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.631169 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" event={"ID":"56aefbfd-00cb-4dee-a618-38b4f5ff60ef","Type":"ContainerDied","Data":"2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966"} Aug 03 16:48:54.631286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.631217 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq" event={"ID":"56aefbfd-00cb-4dee-a618-38b4f5ff60ef","Type":"ContainerDied","Data":"6e134760935a5978cf9193a1983fcf33c47272827dcb0db0112ab3ccef391dd9"} Aug 03 16:48:54.631286 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.631240 2582 scope.go:117] "RemoveContainer" containerID="2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966" Aug 03 16:48:54.640498 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.640482 2582 scope.go:117] "RemoveContainer" containerID="22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad" Aug 03 16:48:54.649047 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.649030 2582 scope.go:117] "RemoveContainer" containerID="2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966" Aug 03 16:48:54.649328 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:48:54.649309 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966\": container with ID starting with 2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966 not found: ID does not exist" containerID="2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966" Aug 03 16:48:54.649381 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.649336 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966"} err="failed to get container status \"2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966\": rpc error: code = NotFound desc = could not find container \"2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966\": container with ID starting with 2ba8e89b80c6d06bc32f999f3f2599d43daeb58736c3885f135c3475e7621966 not found: ID does not exist" Aug 03 16:48:54.649381 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.649352 2582 scope.go:117] "RemoveContainer" containerID="22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad" Aug 03 16:48:54.649559 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:48:54.649545 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad\": container with ID starting with 22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad not found: ID does not exist" containerID="22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad" Aug 03 16:48:54.649600 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.649562 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad"} err="failed to get container status \"22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad\": rpc error: code = NotFound desc = could not find container \"22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad\": container with ID starting with 22ed7982a64fa944b312ab8c55e4ad13f23a24f68680904dc53b2d52ac414bad not found: ID does not exist" Aug 03 16:48:54.660197 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.657784 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq"] Aug 03 16:48:54.665431 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:54.665411 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-deec1a-openshift-default-85684fc87d-nlqxq"] Aug 03 16:48:55.995864 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:48:55.995829 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" path="/var/lib/kubelet/pods/56aefbfd-00cb-4dee-a618-38b4f5ff60ef/volumes" Aug 03 16:49:12.303338 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:12.303308 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78"] Aug 03 16:49:12.303761 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:12.303543 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" containerID="cri-o://775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b" gracePeriod=30 Aug 03 16:49:13.201306 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.201277 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95"] Aug 03 16:49:13.201809 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.201777 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" Aug 03 16:49:13.201809 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.201806 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" Aug 03 16:49:13.201942 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.201898 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" Aug 03 16:49:13.204915 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.204901 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.207693 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.207674 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-0fae8e-openshift-default-dockercfg-59kdc\"" Aug 03 16:49:13.216981 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.216952 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95"] Aug 03 16:49:13.241685 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.241646 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-socket\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.241685 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.241680 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-data\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.241911 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.241696 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn7d7\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-kube-api-access-nn7d7\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.241911 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.241718 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-certs\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.241911 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.241773 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-credential-socket\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.241911 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.241798 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-token\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.241911 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.241847 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-envoy\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.242185 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.241931 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/029af2b3-97b9-42b4-88fb-652e3fe40960-istiod-ca-cert\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.242185 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.241979 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-podinfo\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.342722 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.342682 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-envoy\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.342722 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.342722 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/029af2b3-97b9-42b4-88fb-652e3fe40960-istiod-ca-cert\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.342747 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-podinfo\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.342783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-socket\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.342802 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-data\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.342823 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn7d7\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-kube-api-access-nn7d7\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.342849 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-certs\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.343067 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-credential-socket\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.343107 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-token\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.343326 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-data\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.343466 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-credential-socket\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.343581 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-socket\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.343597 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-certs\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.343679 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.343633 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/029af2b3-97b9-42b4-88fb-652e3fe40960-istiod-ca-cert\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.345157 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.345136 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-envoy\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.345484 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.345468 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-podinfo\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.352444 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.352420 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-token\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.352875 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.352855 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn7d7\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-kube-api-access-nn7d7\") pod \"e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.515050 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.515024 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:13.674214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.674189 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95"] Aug 03 16:49:13.675811 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:49:13.675787 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029af2b3_97b9_42b4_88fb_652e3fe40960.slice/crio-6df78bb18054d86caa4c4f85db0543981c4c959eedda4c3a9cdbd4c3ca5e1c93 WatchSource:0}: Error finding container 6df78bb18054d86caa4c4f85db0543981c4c959eedda4c3a9cdbd4c3ca5e1c93: Status 404 returned error can't find the container with id 6df78bb18054d86caa4c4f85db0543981c4c959eedda4c3a9cdbd4c3ca5e1c93 Aug 03 16:49:13.677894 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.677862 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:49:13.677981 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.677924 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:49:13.677981 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.677951 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:49:13.741336 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:13.741311 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" event={"ID":"029af2b3-97b9-42b4-88fb-652e3fe40960","Type":"ContainerStarted","Data":"6df78bb18054d86caa4c4f85db0543981c4c959eedda4c3a9cdbd4c3ca5e1c93"} Aug 03 16:49:14.748554 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:14.748519 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" event={"ID":"029af2b3-97b9-42b4-88fb-652e3fe40960","Type":"ContainerStarted","Data":"e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17"} Aug 03 16:49:14.771677 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:14.771608 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" podStartSLOduration=1.771595166 podStartE2EDuration="1.771595166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:49:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:49:14.769947515 +0000 UTC m=+2693.328030133" watchObservedRunningTime="2026-08-03 16:49:14.771595166 +0000 UTC m=+2693.329677751" Aug 03 16:49:15.515735 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:15.515702 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:15.520413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:15.520390 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:15.754576 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:15.754546 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:15.755565 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:15.755545 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:17.564096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.564042 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:49:17.684995 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.684961 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11fcf8c-7b30-4a21-9144-44285019e5a1-istiod-ca-cert\") pod \"f11fcf8c-7b30-4a21-9144-44285019e5a1\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " Aug 03 16:49:17.685167 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685022 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-token\") pod \"f11fcf8c-7b30-4a21-9144-44285019e5a1\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " Aug 03 16:49:17.685167 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685045 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-envoy\") pod \"f11fcf8c-7b30-4a21-9144-44285019e5a1\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " Aug 03 16:49:17.685167 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685060 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-socket\") pod \"f11fcf8c-7b30-4a21-9144-44285019e5a1\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " Aug 03 16:49:17.685321 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685164 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-credential-socket\") pod \"f11fcf8c-7b30-4a21-9144-44285019e5a1\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " Aug 03 16:49:17.685321 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685220 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-podinfo\") pod \"f11fcf8c-7b30-4a21-9144-44285019e5a1\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " Aug 03 16:49:17.685321 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685264 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-certs\") pod \"f11fcf8c-7b30-4a21-9144-44285019e5a1\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " Aug 03 16:49:17.685321 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685279 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f11fcf8c-7b30-4a21-9144-44285019e5a1" (UID: "f11fcf8c-7b30-4a21-9144-44285019e5a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:49:17.685519 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685329 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-data\") pod \"f11fcf8c-7b30-4a21-9144-44285019e5a1\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " Aug 03 16:49:17.685519 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685355 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7st8\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-kube-api-access-k7st8\") pod \"f11fcf8c-7b30-4a21-9144-44285019e5a1\" (UID: \"f11fcf8c-7b30-4a21-9144-44285019e5a1\") " Aug 03 16:49:17.685519 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685466 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f11fcf8c-7b30-4a21-9144-44285019e5a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f11fcf8c-7b30-4a21-9144-44285019e5a1" (UID: "f11fcf8c-7b30-4a21-9144-44285019e5a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 16:49:17.685716 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685499 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f11fcf8c-7b30-4a21-9144-44285019e5a1" (UID: "f11fcf8c-7b30-4a21-9144-44285019e5a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:49:17.685716 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685545 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f11fcf8c-7b30-4a21-9144-44285019e5a1" (UID: "f11fcf8c-7b30-4a21-9144-44285019e5a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:49:17.685716 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685643 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "f11fcf8c-7b30-4a21-9144-44285019e5a1" (UID: "f11fcf8c-7b30-4a21-9144-44285019e5a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:49:17.685885 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685753 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:49:17.685885 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685774 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:49:17.685885 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685786 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:49:17.685885 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685801 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:49:17.685885 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.685810 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f11fcf8c-7b30-4a21-9144-44285019e5a1-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:49:17.687291 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.687269 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "f11fcf8c-7b30-4a21-9144-44285019e5a1" (UID: "f11fcf8c-7b30-4a21-9144-44285019e5a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:49:17.687458 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.687437 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-kube-api-access-k7st8" (OuterVolumeSpecName: "kube-api-access-k7st8") pod "f11fcf8c-7b30-4a21-9144-44285019e5a1" (UID: "f11fcf8c-7b30-4a21-9144-44285019e5a1"). InnerVolumeSpecName "kube-api-access-k7st8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:49:17.687692 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.687676 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f11fcf8c-7b30-4a21-9144-44285019e5a1" (UID: "f11fcf8c-7b30-4a21-9144-44285019e5a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:49:17.687760 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.687727 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f11fcf8c-7b30-4a21-9144-44285019e5a1" (UID: "f11fcf8c-7b30-4a21-9144-44285019e5a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 16:49:17.767084 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.767045 2582 generic.go:356] "Generic (PLEG): container finished" podID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerID="775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b" exitCode=0 Aug 03 16:49:17.767273 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.767096 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" Aug 03 16:49:17.767273 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.767133 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" event={"ID":"f11fcf8c-7b30-4a21-9144-44285019e5a1","Type":"ContainerDied","Data":"775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b"} Aug 03 16:49:17.767273 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.767186 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78" event={"ID":"f11fcf8c-7b30-4a21-9144-44285019e5a1","Type":"ContainerDied","Data":"8c8ff496fbc871375a9af3eb9a6897f9feb61c0c483523ee7dd684a1f03331c4"} Aug 03 16:49:17.767273 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.767208 2582 scope.go:117] "RemoveContainer" containerID="775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b" Aug 03 16:49:17.776794 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.776777 2582 scope.go:117] "RemoveContainer" containerID="28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457" Aug 03 16:49:17.784812 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.784797 2582 scope.go:117] "RemoveContainer" containerID="775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b" Aug 03 16:49:17.785044 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:49:17.785026 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b\": container with ID starting with 775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b not found: ID does not exist" containerID="775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b" Aug 03 16:49:17.785096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.785051 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b"} err="failed to get container status \"775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b\": rpc error: code = NotFound desc = could not find container \"775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b\": container with ID starting with 775c39d2ee229083893d4ecd30a4017e2df48230512f8b443917be3dc4c0e48b not found: ID does not exist" Aug 03 16:49:17.785096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.785068 2582 scope.go:117] "RemoveContainer" containerID="28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457" Aug 03 16:49:17.785266 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:49:17.785250 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457\": container with ID starting with 28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457 not found: ID does not exist" containerID="28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457" Aug 03 16:49:17.785306 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.785272 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457"} err="failed to get container status \"28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457\": rpc error: code = NotFound desc = could not find container \"28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457\": container with ID starting with 28f2757ab76a2bfa10ece9cdbf1ebdbda254c21870f33647597027b7e9646457 not found: ID does not exist" Aug 03 16:49:17.786680 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.786665 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:49:17.786727 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.786682 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:49:17.786727 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.786690 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f11fcf8c-7b30-4a21-9144-44285019e5a1-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:49:17.786727 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.786699 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7st8\" (UniqueName: \"kubernetes.io/projected/f11fcf8c-7b30-4a21-9144-44285019e5a1-kube-api-access-k7st8\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:49:17.797065 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.797041 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78"] Aug 03 16:49:17.799171 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.799153 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9ffb05-openshift-default-7855844b6-4fh78"] Aug 03 16:49:17.994775 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:17.994697 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" path="/var/lib/kubelet/pods/f11fcf8c-7b30-4a21-9144-44285019e5a1/volumes" Aug 03 16:49:29.842411 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:29.842372 2582 generic.go:356] "Generic (PLEG): container finished" podID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerID="e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17" exitCode=0 Aug 03 16:49:29.842957 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:29.842432 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" event={"ID":"029af2b3-97b9-42b4-88fb-652e3fe40960","Type":"ContainerDied","Data":"e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17"} Aug 03 16:49:29.843032 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:29.842963 2582 scope.go:117] "RemoveContainer" containerID="e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17" Aug 03 16:49:29.843963 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:29.843935 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:49:29.844067 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:29.844000 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:49:29.844067 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:29.844045 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:49:30.755067 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:30.755025 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:30.854037 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:30.853953 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" event={"ID":"029af2b3-97b9-42b4-88fb-652e3fe40960","Type":"ContainerStarted","Data":"901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd"} Aug 03 16:49:31.515519 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:31.515491 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:31.516554 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:31.516530 2582 patch_prober.go:28] interesting pod/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Aug 03 16:49:31.516660 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:31.516588 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Aug 03 16:49:32.516400 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:32.516364 2582 patch_prober.go:28] interesting pod/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Aug 03 16:49:32.516822 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:32.516416 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Aug 03 16:49:33.516391 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:33.516353 2582 patch_prober.go:28] interesting pod/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Aug 03 16:49:33.516578 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:33.516404 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Aug 03 16:49:34.515952 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:34.515915 2582 patch_prober.go:28] interesting pod/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Aug 03 16:49:34.516144 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:34.515969 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Aug 03 16:49:35.516108 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:35.516072 2582 patch_prober.go:28] interesting pod/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Aug 03 16:49:35.516484 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:35.516137 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Aug 03 16:49:36.519700 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:36.519662 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:36.520151 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:36.519936 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:36.520595 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:36.520576 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 16:49:42.478983 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.478952 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm"] Aug 03 16:49:42.479344 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.479320 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" Aug 03 16:49:42.479344 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.479331 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" Aug 03 16:49:42.479413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.479348 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" Aug 03 16:49:42.479413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.479353 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" Aug 03 16:49:42.479413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.479360 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" Aug 03 16:49:42.479413 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.479365 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" Aug 03 16:49:42.479531 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.479438 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" Aug 03 16:49:42.479531 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.479448 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f11fcf8c-7b30-4a21-9144-44285019e5a1" containerName="istio-proxy" Aug 03 16:49:42.479531 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.479453 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="56aefbfd-00cb-4dee-a618-38b4f5ff60ef" containerName="istio-proxy" Aug 03 16:49:42.482417 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.482403 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.485977 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.485958 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-7055f0-openshift-default-dockercfg-58pln\"" Aug 03 16:49:42.502446 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.502423 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm"] Aug 03 16:49:42.634688 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.634655 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-envoy\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.634878 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.634697 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-socket\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.634878 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.634720 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-credential-socket\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.634878 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.634782 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-token\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.635002 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.634885 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-certs\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.635002 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.634912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-podinfo\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.635002 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.634982 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istiod-ca-cert\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.635002 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.634997 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csxb2\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-kube-api-access-csxb2\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.635171 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.635016 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-data\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.735614 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735528 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-socket\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.735614 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735565 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-credential-socket\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.735614 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735590 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-token\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.735940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735732 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-certs\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.735940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735771 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-podinfo\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.735940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735875 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istiod-ca-cert\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.735940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735898 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csxb2\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-kube-api-access-csxb2\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.735940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735920 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-data\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.736209 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735972 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-socket\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.736209 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.735984 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-envoy\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.736209 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.736062 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-credential-socket\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.736209 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.736095 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-certs\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.736386 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.736222 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-data\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.736553 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.736528 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istiod-ca-cert\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.738170 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.738153 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-envoy\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.738239 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.738221 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-podinfo\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.753159 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.753136 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-token\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.753475 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.753456 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csxb2\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-kube-api-access-csxb2\") pod \"e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.792096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.792065 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:42.943314 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.943285 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm"] Aug 03 16:49:42.944715 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:49:42.944686 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa35e65d_d3ab_4aa0_90d3_18c9825c2c82.slice/crio-15716f1b3040014ad035f145a497eddaa42efc851db5ef4a4da2bb0eb5746ef0 WatchSource:0}: Error finding container 15716f1b3040014ad035f145a497eddaa42efc851db5ef4a4da2bb0eb5746ef0: Status 404 returned error can't find the container with id 15716f1b3040014ad035f145a497eddaa42efc851db5ef4a4da2bb0eb5746ef0 Aug 03 16:49:42.946881 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.946844 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:49:42.946994 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.946907 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:49:42.946994 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:42.946934 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:49:43.926858 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:43.926822 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" event={"ID":"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82","Type":"ContainerStarted","Data":"41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983"} Aug 03 16:49:43.927321 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:43.926873 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" event={"ID":"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82","Type":"ContainerStarted","Data":"15716f1b3040014ad035f145a497eddaa42efc851db5ef4a4da2bb0eb5746ef0"} Aug 03 16:49:43.965236 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:43.965191 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" podStartSLOduration=1.965177262 podStartE2EDuration="1.965177262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:49:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:49:43.965021697 +0000 UTC m=+2722.523104282" watchObservedRunningTime="2026-08-03 16:49:43.965177262 +0000 UTC m=+2722.523259850" Aug 03 16:49:44.793257 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:44.793221 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:44.798416 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:44.798392 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:44.933652 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:44.933623 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:44.934678 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:44.934655 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:49:59.935728 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:59.935690 2582 patch_prober.go:28] interesting pod/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Aug 03 16:49:59.936227 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:49:59.935769 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Aug 03 16:50:00.027253 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:00.027221 2582 generic.go:356] "Generic (PLEG): container finished" podID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerID="41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983" exitCode=0 Aug 03 16:50:00.027421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:00.027292 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" event={"ID":"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82","Type":"ContainerDied","Data":"41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983"} Aug 03 16:50:00.027661 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:00.027647 2582 scope.go:117] "RemoveContainer" containerID="41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983" Aug 03 16:50:00.028614 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:00.028584 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:50:00.028727 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:00.028689 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:50:00.028789 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:00.028735 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:50:01.035944 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:01.035903 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" event={"ID":"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82","Type":"ContainerStarted","Data":"2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f"} Aug 03 16:50:01.792471 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:01.792436 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:50:01.792659 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:01.792640 2582 patch_prober.go:28] interesting pod/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Aug 03 16:50:01.792708 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:01.792681 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Aug 03 16:50:02.792957 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:02.792921 2582 patch_prober.go:28] interesting pod/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Aug 03 16:50:02.793342 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:02.792973 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Aug 03 16:50:03.793210 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:03.793174 2582 patch_prober.go:28] interesting pod/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Aug 03 16:50:03.793599 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:03.793232 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Aug 03 16:50:04.793298 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:04.793258 2582 patch_prober.go:28] interesting pod/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Aug 03 16:50:04.793815 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:04.793322 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Aug 03 16:50:05.796861 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:05.796832 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:50:05.797304 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:05.797101 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:50:05.797735 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:05.797719 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 16:50:43.767240 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.767158 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j"] Aug 03 16:50:43.770086 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.770032 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.774309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.774282 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"kube-root-ca.crt\"" Aug 03 16:50:43.774309 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.774303 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"auth-test-model-7055f0-kserve-self-signed-certs\"" Aug 03 16:50:43.774458 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.774311 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"default-dockercfg-cwtm8\"" Aug 03 16:50:43.774458 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.774357 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-7055f0\"/\"openshift-service-ca.crt\"" Aug 03 16:50:43.782576 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.782550 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j"] Aug 03 16:50:43.810212 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.810174 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tls-certs\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.810212 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.810209 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-model-cache\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.810465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.810279 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shl5q\" (UniqueName: \"kubernetes.io/projected/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-kube-api-access-shl5q\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.810465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.810365 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-home\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.810465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.810414 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tmp-dir\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.810465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.810452 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-dshm\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.911164 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.911129 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-dshm\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.911164 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.911169 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tls-certs\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.911409 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.911184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-model-cache\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.911409 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.911312 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shl5q\" (UniqueName: \"kubernetes.io/projected/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-kube-api-access-shl5q\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.911409 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.911393 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-home\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.911579 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.911460 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tmp-dir\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.911655 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.911594 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-model-cache\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.911753 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.911732 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-home\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.911807 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.911796 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tmp-dir\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.913389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.913365 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-dshm\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.913831 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.913813 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tls-certs\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:43.921192 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:43.921171 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shl5q\" (UniqueName: \"kubernetes.io/projected/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-kube-api-access-shl5q\") pod \"auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:44.079999 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:44.079913 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:44.213739 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:44.213707 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j"] Aug 03 16:50:44.215983 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:50:44.215951 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4c65d9f_be61_4ba0_b76b_e8f930a710a8.slice/crio-eabbd94b8faae493ee7037db6444963b9205850897ce5cd2353b8dbe8b8d8d01 WatchSource:0}: Error finding container eabbd94b8faae493ee7037db6444963b9205850897ce5cd2353b8dbe8b8d8d01: Status 404 returned error can't find the container with id eabbd94b8faae493ee7037db6444963b9205850897ce5cd2353b8dbe8b8d8d01 Aug 03 16:50:44.283570 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:44.283538 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" event={"ID":"a4c65d9f-be61-4ba0-b76b-e8f930a710a8","Type":"ContainerStarted","Data":"e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310"} Aug 03 16:50:44.283729 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:44.283582 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" event={"ID":"a4c65d9f-be61-4ba0-b76b-e8f930a710a8","Type":"ContainerStarted","Data":"eabbd94b8faae493ee7037db6444963b9205850897ce5cd2353b8dbe8b8d8d01"} Aug 03 16:50:44.302188 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:44.302135 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" podStartSLOduration=1.302112958 podStartE2EDuration="1.302112958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:50:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:50:44.300867199 +0000 UTC m=+2782.858949783" watchObservedRunningTime="2026-08-03 16:50:44.302112958 +0000 UTC m=+2782.860195544" Aug 03 16:50:54.080564 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:54.080526 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:54.080564 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:54.080575 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:54.093077 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:54.093056 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:50:54.348331 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:50:54.348241 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:54:24.150439 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:24.150407 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j"] Aug 03 16:54:39.169087 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.169022 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" podUID="a4c65d9f-be61-4ba0-b76b-e8f930a710a8" containerName="main" containerID="cri-o://e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310" gracePeriod=45 Aug 03 16:54:39.419939 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.419886 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:54:39.469825 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.469791 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-home\") pod \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " Aug 03 16:54:39.469997 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.469859 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-model-cache\") pod \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " Aug 03 16:54:39.469997 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.469898 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tmp-dir\") pod \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " Aug 03 16:54:39.469997 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.469951 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shl5q\" (UniqueName: \"kubernetes.io/projected/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-kube-api-access-shl5q\") pod \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " Aug 03 16:54:39.470161 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.470000 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-dshm\") pod \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " Aug 03 16:54:39.470161 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.470036 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tls-certs\") pod \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\" (UID: \"a4c65d9f-be61-4ba0-b76b-e8f930a710a8\") " Aug 03 16:54:39.470161 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.470090 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-home" (OuterVolumeSpecName: "home") pod "a4c65d9f-be61-4ba0-b76b-e8f930a710a8" (UID: "a4c65d9f-be61-4ba0-b76b-e8f930a710a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:54:39.470161 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.470112 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4c65d9f-be61-4ba0-b76b-e8f930a710a8" (UID: "a4c65d9f-be61-4ba0-b76b-e8f930a710a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:54:39.470319 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.470237 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4c65d9f-be61-4ba0-b76b-e8f930a710a8" (UID: "a4c65d9f-be61-4ba0-b76b-e8f930a710a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:54:39.470389 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.470371 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-model-cache\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:54:39.470426 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.470400 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tmp-dir\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:54:39.470426 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.470414 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-home\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:54:39.472234 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.472209 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-dshm" (OuterVolumeSpecName: "dshm") pod "a4c65d9f-be61-4ba0-b76b-e8f930a710a8" (UID: "a4c65d9f-be61-4ba0-b76b-e8f930a710a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:54:39.472334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.472247 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-kube-api-access-shl5q" (OuterVolumeSpecName: "kube-api-access-shl5q") pod "a4c65d9f-be61-4ba0-b76b-e8f930a710a8" (UID: "a4c65d9f-be61-4ba0-b76b-e8f930a710a8"). InnerVolumeSpecName "kube-api-access-shl5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:54:39.472334 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.472287 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4c65d9f-be61-4ba0-b76b-e8f930a710a8" (UID: "a4c65d9f-be61-4ba0-b76b-e8f930a710a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:54:39.571019 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.570986 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-dshm\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:54:39.571019 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.571014 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-tls-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:54:39.571019 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.571024 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shl5q\" (UniqueName: \"kubernetes.io/projected/a4c65d9f-be61-4ba0-b76b-e8f930a710a8-kube-api-access-shl5q\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:54:39.631522 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.631490 2582 generic.go:356] "Generic (PLEG): container finished" podID="a4c65d9f-be61-4ba0-b76b-e8f930a710a8" containerID="e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310" exitCode=0 Aug 03 16:54:39.631712 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.631533 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" Aug 03 16:54:39.631712 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.631556 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" event={"ID":"a4c65d9f-be61-4ba0-b76b-e8f930a710a8","Type":"ContainerDied","Data":"e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310"} Aug 03 16:54:39.631712 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.631585 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j" event={"ID":"a4c65d9f-be61-4ba0-b76b-e8f930a710a8","Type":"ContainerDied","Data":"eabbd94b8faae493ee7037db6444963b9205850897ce5cd2353b8dbe8b8d8d01"} Aug 03 16:54:39.631712 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.631602 2582 scope.go:117] "RemoveContainer" containerID="e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310" Aug 03 16:54:39.642426 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.642405 2582 scope.go:117] "RemoveContainer" containerID="e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310" Aug 03 16:54:39.642725 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:54:39.642705 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310\": container with ID starting with e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310 not found: ID does not exist" containerID="e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310" Aug 03 16:54:39.642801 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.642732 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310"} err="failed to get container status \"e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310\": rpc error: code = NotFound desc = could not find container \"e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310\": container with ID starting with e9e713c85baaa1d1a840d3359561d2cf6a6f7e2e182890182cb84085da442310 not found: ID does not exist" Aug 03 16:54:39.654860 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.654827 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j"] Aug 03 16:54:39.661308 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.661285 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-7055f0/auth-test-model-7055f0-kserve-6f8458fdf9-vxq7j"] Aug 03 16:54:39.995706 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:54:39.995672 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4c65d9f-be61-4ba0-b76b-e8f930a710a8" path="/var/lib/kubelet/pods/a4c65d9f-be61-4ba0-b76b-e8f930a710a8/volumes" Aug 03 16:56:33.157613 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.157570 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h"] Aug 03 16:56:33.158268 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.158099 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c65d9f-be61-4ba0-b76b-e8f930a710a8" containerName="main" Aug 03 16:56:33.158268 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.158118 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c65d9f-be61-4ba0-b76b-e8f930a710a8" containerName="main" Aug 03 16:56:33.158268 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.158244 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4c65d9f-be61-4ba0-b76b-e8f930a710a8" containerName="main" Aug 03 16:56:33.161341 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.161320 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.165374 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.165352 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0fae8e\"/\"rate-test-model-0fae8e-kserve-self-signed-certs\"" Aug 03 16:56:33.165488 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.165397 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0fae8e\"/\"kube-root-ca.crt\"" Aug 03 16:56:33.165488 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.165356 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0fae8e\"/\"default-dockercfg-whcnp\"" Aug 03 16:56:33.165488 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.165352 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0fae8e\"/\"openshift-service-ca.crt\"" Aug 03 16:56:33.172375 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.172351 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h"] Aug 03 16:56:33.231327 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.231289 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-tmp-dir\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.231510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.231348 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-dshm\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.231510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.231375 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-home\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.231510 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.231423 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9fj5\" (UniqueName: \"kubernetes.io/projected/68cd53be-7b90-4185-9182-29e3664e2f18-kube-api-access-g9fj5\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.231666 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.231534 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-model-cache\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.231666 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.231591 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cd53be-7b90-4185-9182-29e3664e2f18-tls-certs\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.332937 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.332904 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cd53be-7b90-4185-9182-29e3664e2f18-tls-certs\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.333146 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.332954 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-tmp-dir\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.333146 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.332973 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-dshm\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.333146 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.332991 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-home\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.333146 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.333016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9fj5\" (UniqueName: \"kubernetes.io/projected/68cd53be-7b90-4185-9182-29e3664e2f18-kube-api-access-g9fj5\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.333146 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.333094 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-model-cache\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.333429 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.333372 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-tmp-dir\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.333471 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.333432 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-home\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.333506 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.333488 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-model-cache\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.335221 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.335200 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-dshm\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.335595 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.335572 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cd53be-7b90-4185-9182-29e3664e2f18-tls-certs\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.343923 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.343903 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9fj5\" (UniqueName: \"kubernetes.io/projected/68cd53be-7b90-4185-9182-29e3664e2f18-kube-api-access-g9fj5\") pod \"rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.471113 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.471019 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:33.608589 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:33.608565 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h"] Aug 03 16:56:33.610046 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:56:33.610022 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68cd53be_7b90_4185_9182_29e3664e2f18.slice/crio-4c522595a8f3a6e17bbd7f0c3a2a8613ac76677569dd495d5b26d51de9535bf1 WatchSource:0}: Error finding container 4c522595a8f3a6e17bbd7f0c3a2a8613ac76677569dd495d5b26d51de9535bf1: Status 404 returned error can't find the container with id 4c522595a8f3a6e17bbd7f0c3a2a8613ac76677569dd495d5b26d51de9535bf1 Aug 03 16:56:34.274806 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:34.274769 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" event={"ID":"68cd53be-7b90-4185-9182-29e3664e2f18","Type":"ContainerStarted","Data":"21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385"} Aug 03 16:56:34.274806 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:34.274809 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" event={"ID":"68cd53be-7b90-4185-9182-29e3664e2f18","Type":"ContainerStarted","Data":"4c522595a8f3a6e17bbd7f0c3a2a8613ac76677569dd495d5b26d51de9535bf1"} Aug 03 16:56:34.292368 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:34.292322 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" podStartSLOduration=1.292305934 podStartE2EDuration="1.292305934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:56:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:56:34.291711121 +0000 UTC m=+3132.849793709" watchObservedRunningTime="2026-08-03 16:56:34.292305934 +0000 UTC m=+3132.850388520" Aug 03 16:56:43.471982 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:43.471949 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:43.472411 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:43.472013 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:43.484243 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:43.484221 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:56:44.341935 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:56:44.341906 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:58:25.894892 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:25.894822 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h"] Aug 03 16:58:40.913214 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:40.913150 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" podUID="68cd53be-7b90-4185-9182-29e3664e2f18" containerName="main" containerID="cri-o://21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385" gracePeriod=45 Aug 03 16:58:41.165394 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.165345 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:58:41.252787 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.252763 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-tmp-dir\") pod \"68cd53be-7b90-4185-9182-29e3664e2f18\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " Aug 03 16:58:41.252906 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.252812 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-home\") pod \"68cd53be-7b90-4185-9182-29e3664e2f18\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " Aug 03 16:58:41.252906 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.252847 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-dshm\") pod \"68cd53be-7b90-4185-9182-29e3664e2f18\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " Aug 03 16:58:41.252906 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.252878 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-model-cache\") pod \"68cd53be-7b90-4185-9182-29e3664e2f18\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " Aug 03 16:58:41.252906 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.252900 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cd53be-7b90-4185-9182-29e3664e2f18-tls-certs\") pod \"68cd53be-7b90-4185-9182-29e3664e2f18\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " Aug 03 16:58:41.253065 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.252939 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9fj5\" (UniqueName: \"kubernetes.io/projected/68cd53be-7b90-4185-9182-29e3664e2f18-kube-api-access-g9fj5\") pod \"68cd53be-7b90-4185-9182-29e3664e2f18\" (UID: \"68cd53be-7b90-4185-9182-29e3664e2f18\") " Aug 03 16:58:41.253116 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.253054 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-home" (OuterVolumeSpecName: "home") pod "68cd53be-7b90-4185-9182-29e3664e2f18" (UID: "68cd53be-7b90-4185-9182-29e3664e2f18"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:58:41.253116 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.253061 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "68cd53be-7b90-4185-9182-29e3664e2f18" (UID: "68cd53be-7b90-4185-9182-29e3664e2f18"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:58:41.253284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.253135 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-model-cache" (OuterVolumeSpecName: "model-cache") pod "68cd53be-7b90-4185-9182-29e3664e2f18" (UID: "68cd53be-7b90-4185-9182-29e3664e2f18"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:58:41.253284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.253166 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-tmp-dir\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:58:41.253284 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.253186 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-home\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:58:41.255036 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.255012 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68cd53be-7b90-4185-9182-29e3664e2f18-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "68cd53be-7b90-4185-9182-29e3664e2f18" (UID: "68cd53be-7b90-4185-9182-29e3664e2f18"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 03 16:58:41.255138 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.255049 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-dshm" (OuterVolumeSpecName: "dshm") pod "68cd53be-7b90-4185-9182-29e3664e2f18" (UID: "68cd53be-7b90-4185-9182-29e3664e2f18"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 16:58:41.255138 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.255068 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68cd53be-7b90-4185-9182-29e3664e2f18-kube-api-access-g9fj5" (OuterVolumeSpecName: "kube-api-access-g9fj5") pod "68cd53be-7b90-4185-9182-29e3664e2f18" (UID: "68cd53be-7b90-4185-9182-29e3664e2f18"). InnerVolumeSpecName "kube-api-access-g9fj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 16:58:41.353782 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.353757 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-dshm\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:58:41.353782 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.353781 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68cd53be-7b90-4185-9182-29e3664e2f18-model-cache\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:58:41.353916 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.353796 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68cd53be-7b90-4185-9182-29e3664e2f18-tls-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:58:41.353916 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.353808 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9fj5\" (UniqueName: \"kubernetes.io/projected/68cd53be-7b90-4185-9182-29e3664e2f18-kube-api-access-g9fj5\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 16:58:41.997502 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.997469 2582 generic.go:356] "Generic (PLEG): container finished" podID="68cd53be-7b90-4185-9182-29e3664e2f18" containerID="21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385" exitCode=0 Aug 03 16:58:41.997931 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.997572 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" Aug 03 16:58:41.998663 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.998638 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" event={"ID":"68cd53be-7b90-4185-9182-29e3664e2f18","Type":"ContainerDied","Data":"21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385"} Aug 03 16:58:41.998717 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.998684 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h" event={"ID":"68cd53be-7b90-4185-9182-29e3664e2f18","Type":"ContainerDied","Data":"4c522595a8f3a6e17bbd7f0c3a2a8613ac76677569dd495d5b26d51de9535bf1"} Aug 03 16:58:41.998717 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:41.998710 2582 scope.go:117] "RemoveContainer" containerID="21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385" Aug 03 16:58:42.008845 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:42.008826 2582 scope.go:117] "RemoveContainer" containerID="21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385" Aug 03 16:58:42.009197 ip-10-0-142-140 kubenswrapper[2582]: E0803 16:58:42.009173 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385\": container with ID starting with 21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385 not found: ID does not exist" containerID="21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385" Aug 03 16:58:42.009281 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:42.009203 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385"} err="failed to get container status \"21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385\": rpc error: code = NotFound desc = could not find container \"21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385\": container with ID starting with 21271818c1481e2fe6dfd2a5f05afd1f5b1a4d2629e5d0b4f6de78d17670f385 not found: ID does not exist" Aug 03 16:58:42.023561 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:42.023540 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h"] Aug 03 16:58:42.029829 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:42.029808 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0fae8e/rate-test-model-0fae8e-kserve-7cbb86cc75-5r52h"] Aug 03 16:58:43.995951 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:43.995909 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68cd53be-7b90-4185-9182-29e3664e2f18" path="/var/lib/kubelet/pods/68cd53be-7b90-4185-9182-29e3664e2f18/volumes" Aug 03 16:58:46.926983 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.926954 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p"] Aug 03 16:58:46.927419 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.927332 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68cd53be-7b90-4185-9182-29e3664e2f18" containerName="main" Aug 03 16:58:46.927419 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.927342 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="68cd53be-7b90-4185-9182-29e3664e2f18" containerName="main" Aug 03 16:58:46.927419 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.927417 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="68cd53be-7b90-4185-9182-29e3664e2f18" containerName="main" Aug 03 16:58:46.932130 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.932108 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:46.935209 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.935190 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-0e4b78-openshift-default-dockercfg-bxnt4\"" Aug 03 16:58:46.943788 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.943765 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p"] Aug 03 16:58:46.997857 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.997833 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-data\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:46.997965 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.997862 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-certs\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:46.997965 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.997883 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-envoy\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:46.997965 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.997926 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-credential-socket\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:46.997965 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.997961 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff37a45-e0f2-4cf1-be98-47496985a035-istiod-ca-cert\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:46.998134 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.997985 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff37a45-e0f2-4cf1-be98-47496985a035-istio-podinfo\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:46.998134 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.998022 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-istio-token\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:46.998134 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.998044 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pggml\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-kube-api-access-pggml\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:46.998134 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:46.998072 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-socket\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099080 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099049 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-data\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099085 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-certs\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099111 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-envoy\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099138 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-credential-socket\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099168 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff37a45-e0f2-4cf1-be98-47496985a035-istiod-ca-cert\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099189 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff37a45-e0f2-4cf1-be98-47496985a035-istio-podinfo\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099270 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099235 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-istio-token\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099296 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pggml\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-kube-api-access-pggml\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099373 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-socket\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099517 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-data\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099610 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099570 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-certs\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099857 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099711 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-credential-socket\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.099951 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.099932 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-socket\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.100096 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.100075 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff37a45-e0f2-4cf1-be98-47496985a035-istiod-ca-cert\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.101431 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.101414 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-envoy\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.101787 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.101765 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff37a45-e0f2-4cf1-be98-47496985a035-istio-podinfo\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.109508 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.109485 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pggml\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-kube-api-access-pggml\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.110066 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.110048 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-istio-token\") pod \"e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.242818 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.242764 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:47.380350 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.380321 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p"] Aug 03 16:58:47.381576 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:58:47.381552 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff37a45_e0f2_4cf1_be98_47496985a035.slice/crio-2b64fb413cfb2b4c9750a140225793f9df5174168c676831c15024b00742da09 WatchSource:0}: Error finding container 2b64fb413cfb2b4c9750a140225793f9df5174168c676831c15024b00742da09: Status 404 returned error can't find the container with id 2b64fb413cfb2b4c9750a140225793f9df5174168c676831c15024b00742da09 Aug 03 16:58:47.383841 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.383808 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:58:47.383921 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.383877 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:58:47.383921 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:47.383903 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:58:48.033559 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:48.033512 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" event={"ID":"aff37a45-e0f2-4cf1-be98-47496985a035","Type":"ContainerStarted","Data":"f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a"} Aug 03 16:58:48.033559 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:48.033568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" event={"ID":"aff37a45-e0f2-4cf1-be98-47496985a035","Type":"ContainerStarted","Data":"2b64fb413cfb2b4c9750a140225793f9df5174168c676831c15024b00742da09"} Aug 03 16:58:48.056450 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:48.056396 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" podStartSLOduration=2.056382966 podStartE2EDuration="2.056382966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:58:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:58:48.054859253 +0000 UTC m=+3266.612941838" watchObservedRunningTime="2026-08-03 16:58:48.056382966 +0000 UTC m=+3266.614465551" Aug 03 16:58:48.242956 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:48.242924 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:48.247815 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:48.247791 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:49.039379 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:49.039350 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:58:49.040182 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:58:49.040164 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:59:04.040599 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:04.040569 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 03 16:59:04.041030 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:04.040645 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 03 16:59:04.133611 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:04.133585 2582 generic.go:356] "Generic (PLEG): container finished" podID="aff37a45-e0f2-4cf1-be98-47496985a035" containerID="f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a" exitCode=0 Aug 03 16:59:04.133775 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:04.133657 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" event={"ID":"aff37a45-e0f2-4cf1-be98-47496985a035","Type":"ContainerDied","Data":"f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a"} Aug 03 16:59:04.134035 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:04.134020 2582 scope.go:117] "RemoveContainer" containerID="f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a" Aug 03 16:59:04.135175 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:04.135147 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:59:04.135236 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:04.135221 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:59:04.135272 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:04.135258 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:59:05.146800 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:05.146759 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" event={"ID":"aff37a45-e0f2-4cf1-be98-47496985a035","Type":"ContainerStarted","Data":"0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd"} Aug 03 16:59:05.243108 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:05.243075 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:59:05.243645 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:05.243327 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 03 16:59:05.243645 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:05.243375 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 03 16:59:06.243879 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:06.243841 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 03 16:59:06.244385 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:06.243900 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 03 16:59:07.244083 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:07.244047 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 03 16:59:07.244683 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:07.244648 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 03 16:59:08.243692 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:08.243654 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 03 16:59:08.243909 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:08.243731 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 03 16:59:09.243419 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:09.243376 2582 patch_prober.go:28] interesting pod/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body= Aug 03 16:59:09.243924 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:09.243451 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" Aug 03 16:59:10.247477 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:10.247447 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:59:10.247940 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:10.247860 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:59:10.248858 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:10.248839 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 16:59:16.023155 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.023121 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg"] Aug 03 16:59:16.025569 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.025552 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.028421 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.028400 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-cf8ed9-openshift-default-dockercfg-d4m5h\"" Aug 03 16:59:16.041139 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.041117 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg"] Aug 03 16:59:16.066350 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.066329 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-credential-socket\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.066448 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.066374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-istio-token\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.066448 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.066404 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-certs\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.066522 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.066480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-envoy\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.066645 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.066611 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfvtb\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-kube-api-access-cfvtb\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.066725 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.066706 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-socket\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.066773 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.066740 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-data\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.066773 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.066764 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423be317-b793-47e8-8ae8-ba18dddf6223-istio-podinfo\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.066843 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.066817 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423be317-b793-47e8-8ae8-ba18dddf6223-istiod-ca-cert\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.167717 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.167680 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvtb\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-kube-api-access-cfvtb\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.167717 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.167721 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-socket\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.167977 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.167737 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-data\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.167977 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.167848 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423be317-b793-47e8-8ae8-ba18dddf6223-istio-podinfo\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.167977 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.167901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423be317-b793-47e8-8ae8-ba18dddf6223-istiod-ca-cert\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.167977 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.167945 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-credential-socket\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.168206 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.168025 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-istio-token\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.168206 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.168062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-certs\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.168206 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.168123 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-envoy\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.168206 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.168158 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-socket\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.168420 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.168313 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-data\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.168420 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.168345 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-credential-socket\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.168564 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.168546 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-certs\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.168670 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.168647 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423be317-b793-47e8-8ae8-ba18dddf6223-istiod-ca-cert\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.170459 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.170434 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-envoy\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.170607 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.170583 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423be317-b793-47e8-8ae8-ba18dddf6223-istio-podinfo\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.177536 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.177510 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-istio-token\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.177851 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.177834 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfvtb\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-kube-api-access-cfvtb\") pod \"e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.335985 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.335892 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:16.476454 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.476427 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg"] Aug 03 16:59:16.477632 ip-10-0-142-140 kubenswrapper[2582]: W0803 16:59:16.477597 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod423be317_b793_47e8_8ae8_ba18dddf6223.slice/crio-b0cb0531c0d15ad15c0ab81a6c4aeb856a124690ce3de3b1ec53851f675abfb6 WatchSource:0}: Error finding container b0cb0531c0d15ad15c0ab81a6c4aeb856a124690ce3de3b1ec53851f675abfb6: Status 404 returned error can't find the container with id b0cb0531c0d15ad15c0ab81a6c4aeb856a124690ce3de3b1ec53851f675abfb6 Aug 03 16:59:16.479889 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.479858 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:59:16.479978 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.479921 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:59:16.479978 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:16.479952 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:59:17.222634 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:17.222598 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" event={"ID":"423be317-b793-47e8-8ae8-ba18dddf6223","Type":"ContainerStarted","Data":"ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f"} Aug 03 16:59:17.223067 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:17.222654 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" event={"ID":"423be317-b793-47e8-8ae8-ba18dddf6223","Type":"ContainerStarted","Data":"b0cb0531c0d15ad15c0ab81a6c4aeb856a124690ce3de3b1ec53851f675abfb6"} Aug 03 16:59:17.253491 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:17.253446 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" podStartSLOduration=2.253431964 podStartE2EDuration="2.253431964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 16:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 16:59:17.250006051 +0000 UTC m=+3295.808088636" watchObservedRunningTime="2026-08-03 16:59:17.253431964 +0000 UTC m=+3295.811514549" Aug 03 16:59:17.336646 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:17.336608 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:17.342133 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:17.342108 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:18.234985 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:18.234954 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:18.235904 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:18.235870 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:33.236931 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:33.236901 2582 patch_prober.go:28] interesting pod/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Aug 03 16:59:33.237341 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:33.236958 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Aug 03 16:59:33.329347 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:33.329312 2582 generic.go:356] "Generic (PLEG): container finished" podID="423be317-b793-47e8-8ae8-ba18dddf6223" containerID="ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f" exitCode=0 Aug 03 16:59:33.329538 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:33.329387 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" event={"ID":"423be317-b793-47e8-8ae8-ba18dddf6223","Type":"ContainerDied","Data":"ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f"} Aug 03 16:59:33.329869 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:33.329851 2582 scope.go:117] "RemoveContainer" containerID="ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f" Aug 03 16:59:33.330874 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:33.330847 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:59:33.330944 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:33.330910 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:59:33.330944 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:33.330939 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 16:59:34.341142 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:34.341098 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" event={"ID":"423be317-b793-47e8-8ae8-ba18dddf6223","Type":"ContainerStarted","Data":"f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053"} Aug 03 16:59:35.336377 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:35.336338 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:35.336599 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:35.336481 2582 patch_prober.go:28] interesting pod/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Aug 03 16:59:35.336599 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:35.336527 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Aug 03 16:59:36.336465 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:36.336432 2582 patch_prober.go:28] interesting pod/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Aug 03 16:59:36.337071 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:36.337038 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Aug 03 16:59:37.336522 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:37.336482 2582 patch_prober.go:28] interesting pod/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Aug 03 16:59:37.337051 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:37.336552 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Aug 03 16:59:38.336716 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:38.336678 2582 patch_prober.go:28] interesting pod/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body= Aug 03 16:59:38.337233 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:38.336743 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" Aug 03 16:59:39.340584 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:39.340558 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:39.341048 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:39.341031 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 16:59:39.341698 ip-10-0-142-140 kubenswrapper[2582]: I0803 16:59:39.341679 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 17:00:35.084347 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.084317 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8"] Aug 03 17:00:35.087864 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.087843 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.090863 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.090842 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-4df7e0-openshift-default-dockercfg-7ggjt\"" Aug 03 17:00:35.100520 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.100496 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8"] Aug 03 17:00:35.254553 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.254517 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-data\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.254784 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.254581 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-certs\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.254784 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.254614 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.254784 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.254707 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-token\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.254784 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.254756 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-envoy\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.254997 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.254784 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-socket\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.254997 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.254821 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-podinfo\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.254997 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.254887 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks529\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-kube-api-access-ks529\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.254997 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.254924 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-credential-socket\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356239 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356160 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks529\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-kube-api-access-ks529\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356239 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356203 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-credential-socket\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356434 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356244 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-data\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356434 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356298 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-certs\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356434 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356324 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356434 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356354 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-token\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356434 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356394 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-envoy\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356434 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356422 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-socket\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356792 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356454 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-podinfo\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.356964 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356936 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-socket\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.357069 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.356969 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-certs\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.357069 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.357033 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-data\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.357187 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.357084 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-credential-socket\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.357736 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.357717 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.358980 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.358960 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-envoy\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.359035 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.358963 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-podinfo\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.367559 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.367534 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-token\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.367728 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.367708 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks529\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-kube-api-access-ks529\") pod \"e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.399267 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.399243 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:35.541117 ip-10-0-142-140 kubenswrapper[2582]: W0803 17:00:35.541087 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaea84f1b_0f91_4e29_9a58_cac3e316b88a.slice/crio-09a3e80c5f5b3f81ef619d2b2398343695d1adf3f8ecc765871302c21b58886a WatchSource:0}: Error finding container 09a3e80c5f5b3f81ef619d2b2398343695d1adf3f8ecc765871302c21b58886a: Status 404 returned error can't find the container with id 09a3e80c5f5b3f81ef619d2b2398343695d1adf3f8ecc765871302c21b58886a Aug 03 17:00:35.542030 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.541944 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8"] Aug 03 17:00:35.543908 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.543880 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 17:00:35.544004 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.543948 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 17:00:35.544004 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.543981 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 17:00:35.693984 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.693940 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" event={"ID":"aea84f1b-0f91-4e29-9a58-cac3e316b88a","Type":"ContainerStarted","Data":"24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86"} Aug 03 17:00:35.694169 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.694012 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" event={"ID":"aea84f1b-0f91-4e29-9a58-cac3e316b88a","Type":"ContainerStarted","Data":"09a3e80c5f5b3f81ef619d2b2398343695d1adf3f8ecc765871302c21b58886a"} Aug 03 17:00:35.720103 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:35.720053 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" podStartSLOduration=0.720038568 podStartE2EDuration="720.038568ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:00:35.718112201 +0000 UTC m=+3374.276194799" watchObservedRunningTime="2026-08-03 17:00:35.720038568 +0000 UTC m=+3374.278121152" Aug 03 17:00:36.399447 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:36.399416 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:36.403794 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:36.403767 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:36.705715 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:36.705625 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:36.706326 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:36.706307 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:52.162768 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:52.162128 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": read tcp 10.134.0.2:45596->10.134.0.70:15021: read: connection reset by peer" start-of-body= Aug 03 17:00:52.162768 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:52.162199 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": read tcp 10.134.0.2:45596->10.134.0.70:15021: read: connection reset by peer" Aug 03 17:00:52.818705 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:52.818674 2582 generic.go:356] "Generic (PLEG): container finished" podID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerID="24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86" exitCode=0 Aug 03 17:00:52.818872 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:52.818720 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" event={"ID":"aea84f1b-0f91-4e29-9a58-cac3e316b88a","Type":"ContainerDied","Data":"24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86"} Aug 03 17:00:52.819070 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:52.819057 2582 scope.go:117] "RemoveContainer" containerID="24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86" Aug 03 17:00:52.820106 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:52.820079 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 17:00:52.820161 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:52.820143 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 17:00:52.820195 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:52.820173 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Aug 03 17:00:53.826100 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:53.826070 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" event={"ID":"aea84f1b-0f91-4e29-9a58-cac3e316b88a","Type":"ContainerStarted","Data":"30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a"} Aug 03 17:00:54.400149 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:54.400115 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:54.400319 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:54.400257 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body= Aug 03 17:00:54.400375 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:54.400310 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" Aug 03 17:00:55.399499 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:55.399459 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body= Aug 03 17:00:55.399923 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:55.399523 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" Aug 03 17:00:56.399799 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:56.399767 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body= Aug 03 17:00:56.400192 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:56.399817 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" Aug 03 17:00:57.399899 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:57.399865 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body= Aug 03 17:00:57.400284 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:57.399928 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" Aug 03 17:00:58.400249 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:58.400216 2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body= Aug 03 17:00:58.400663 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:58.400269 2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" Aug 03 17:00:59.403294 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:59.403264 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:59.403742 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:59.403646 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:00:59.404274 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:00:59.404247 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:01:16.772501 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:16.772424 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8"] Aug 03 17:01:16.773022 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:16.772644 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" containerID="cri-o://30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a" gracePeriod=30 Aug 03 17:01:22.022294 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.022272 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:01:22.026136 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.026112 2582 generic.go:356] "Generic (PLEG): container finished" podID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerID="30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a" exitCode=0 Aug 03 17:01:22.026267 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.026151 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" Aug 03 17:01:22.026267 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.026155 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" event={"ID":"aea84f1b-0f91-4e29-9a58-cac3e316b88a","Type":"ContainerDied","Data":"30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a"} Aug 03 17:01:22.026267 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.026217 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8" event={"ID":"aea84f1b-0f91-4e29-9a58-cac3e316b88a","Type":"ContainerDied","Data":"09a3e80c5f5b3f81ef619d2b2398343695d1adf3f8ecc765871302c21b58886a"} Aug 03 17:01:22.026267 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.026250 2582 scope.go:117] "RemoveContainer" containerID="30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a" Aug 03 17:01:22.034681 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.034667 2582 scope.go:117] "RemoveContainer" containerID="24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86" Aug 03 17:01:22.042761 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.042691 2582 scope.go:117] "RemoveContainer" containerID="30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a" Aug 03 17:01:22.043227 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:01:22.043102 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a\": container with ID starting with 30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a not found: ID does not exist" containerID="30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a" Aug 03 17:01:22.043227 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.043136 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a"} err="failed to get container status \"30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a\": rpc error: code = NotFound desc = could not find container \"30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a\": container with ID starting with 30c903fd753cc9f32b006c325a804f57a6d65c17fcc2752d3159ddf2952cc01a not found: ID does not exist" Aug 03 17:01:22.043227 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.043156 2582 scope.go:117] "RemoveContainer" containerID="24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86" Aug 03 17:01:22.043550 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:01:22.043510 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86\": container with ID starting with 24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86 not found: ID does not exist" containerID="24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86" Aug 03 17:01:22.043655 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.043547 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86"} err="failed to get container status \"24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86\": rpc error: code = NotFound desc = could not find container \"24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86\": container with ID starting with 24203726da8591d0b775063ce7ebfc6fdd302fc21470d97e6f73adf1dce08c86 not found: ID does not exist" Aug 03 17:01:22.080850 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.080819 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istiod-ca-cert\") pod \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " Aug 03 17:01:22.080938 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.080865 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-data\") pod \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " Aug 03 17:01:22.080938 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.080886 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-token\") pod \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " Aug 03 17:01:22.081030 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.080984 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-credential-socket\") pod \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " Aug 03 17:01:22.081030 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081026 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-envoy\") pod \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " Aug 03 17:01:22.081127 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081113 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-podinfo\") pod \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " Aug 03 17:01:22.081177 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081161 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-certs\") pod \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " Aug 03 17:01:22.081233 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081173 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-data" (OuterVolumeSpecName: "istio-data") pod "aea84f1b-0f91-4e29-9a58-cac3e316b88a" (UID: "aea84f1b-0f91-4e29-9a58-cac3e316b88a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:01:22.081233 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081185 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aea84f1b-0f91-4e29-9a58-cac3e316b88a" (UID: "aea84f1b-0f91-4e29-9a58-cac3e316b88a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 17:01:22.081233 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081200 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aea84f1b-0f91-4e29-9a58-cac3e316b88a" (UID: "aea84f1b-0f91-4e29-9a58-cac3e316b88a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:01:22.081233 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081218 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ks529\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-kube-api-access-ks529\") pod \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " Aug 03 17:01:22.081441 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081247 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-socket\") pod \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\" (UID: \"aea84f1b-0f91-4e29-9a58-cac3e316b88a\") " Aug 03 17:01:22.081441 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081407 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aea84f1b-0f91-4e29-9a58-cac3e316b88a" (UID: "aea84f1b-0f91-4e29-9a58-cac3e316b88a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:01:22.081591 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081571 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aea84f1b-0f91-4e29-9a58-cac3e316b88a" (UID: "aea84f1b-0f91-4e29-9a58-cac3e316b88a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:01:22.081715 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081579 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:01:22.081715 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081644 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:01:22.081715 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081660 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:01:22.081715 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.081675 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:01:22.083165 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.083143 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aea84f1b-0f91-4e29-9a58-cac3e316b88a" (UID: "aea84f1b-0f91-4e29-9a58-cac3e316b88a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 17:01:22.083165 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.083150 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-kube-api-access-ks529" (OuterVolumeSpecName: "kube-api-access-ks529") pod "aea84f1b-0f91-4e29-9a58-cac3e316b88a" (UID: "aea84f1b-0f91-4e29-9a58-cac3e316b88a"). InnerVolumeSpecName "kube-api-access-ks529". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:01:22.083312 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.083261 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-token" (OuterVolumeSpecName: "istio-token") pod "aea84f1b-0f91-4e29-9a58-cac3e316b88a" (UID: "aea84f1b-0f91-4e29-9a58-cac3e316b88a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:01:22.083312 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.083282 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aea84f1b-0f91-4e29-9a58-cac3e316b88a" (UID: "aea84f1b-0f91-4e29-9a58-cac3e316b88a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:01:22.183069 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.183038 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:01:22.183069 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.183070 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:01:22.183069 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.183079 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aea84f1b-0f91-4e29-9a58-cac3e316b88a-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:01:22.183069 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.183088 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks529\" (UniqueName: \"kubernetes.io/projected/aea84f1b-0f91-4e29-9a58-cac3e316b88a-kube-api-access-ks529\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:01:22.183287 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.183097 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aea84f1b-0f91-4e29-9a58-cac3e316b88a-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:01:22.351235 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.351213 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8"] Aug 03 17:01:22.355473 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:22.355454 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-4df7e0-openshift-default-666f646c76-f9qb8"] Aug 03 17:01:24.001683 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:01:24.001648 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" path="/var/lib/kubelet/pods/aea84f1b-0f91-4e29-9a58-cac3e316b88a/volumes" Aug 03 17:02:41.403417 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:41.403382 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p"] Aug 03 17:02:41.404015 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:41.403603 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" containerID="cri-o://0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd" gracePeriod=30 Aug 03 17:02:46.652168 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.652142 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 17:02:46.787603 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787575 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-data\") pod \"aff37a45-e0f2-4cf1-be98-47496985a035\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " Aug 03 17:02:46.787799 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787611 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-certs\") pod \"aff37a45-e0f2-4cf1-be98-47496985a035\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " Aug 03 17:02:46.787799 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787661 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-socket\") pod \"aff37a45-e0f2-4cf1-be98-47496985a035\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " Aug 03 17:02:46.787799 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787711 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pggml\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-kube-api-access-pggml\") pod \"aff37a45-e0f2-4cf1-be98-47496985a035\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " Aug 03 17:02:46.787799 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787764 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff37a45-e0f2-4cf1-be98-47496985a035-istiod-ca-cert\") pod \"aff37a45-e0f2-4cf1-be98-47496985a035\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " Aug 03 17:02:46.788073 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787815 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-credential-socket\") pod \"aff37a45-e0f2-4cf1-be98-47496985a035\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " Aug 03 17:02:46.788073 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787857 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff37a45-e0f2-4cf1-be98-47496985a035-istio-podinfo\") pod \"aff37a45-e0f2-4cf1-be98-47496985a035\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " Aug 03 17:02:46.788073 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787896 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-data" (OuterVolumeSpecName: "istio-data") pod "aff37a45-e0f2-4cf1-be98-47496985a035" (UID: "aff37a45-e0f2-4cf1-be98-47496985a035"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:02:46.788073 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787910 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-envoy\") pod \"aff37a45-e0f2-4cf1-be98-47496985a035\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " Aug 03 17:02:46.788073 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787946 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-istio-token\") pod \"aff37a45-e0f2-4cf1-be98-47496985a035\" (UID: \"aff37a45-e0f2-4cf1-be98-47496985a035\") " Aug 03 17:02:46.788073 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787970 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aff37a45-e0f2-4cf1-be98-47496985a035" (UID: "aff37a45-e0f2-4cf1-be98-47496985a035"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:02:46.788073 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.787988 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aff37a45-e0f2-4cf1-be98-47496985a035" (UID: "aff37a45-e0f2-4cf1-be98-47496985a035"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:02:46.788725 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.788068 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aff37a45-e0f2-4cf1-be98-47496985a035" (UID: "aff37a45-e0f2-4cf1-be98-47496985a035"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:02:46.788725 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.788153 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff37a45-e0f2-4cf1-be98-47496985a035-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aff37a45-e0f2-4cf1-be98-47496985a035" (UID: "aff37a45-e0f2-4cf1-be98-47496985a035"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 17:02:46.788725 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.788307 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:02:46.788725 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.788327 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:02:46.788725 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.788339 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:02:46.788725 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.788351 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:02:46.788725 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.788363 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff37a45-e0f2-4cf1-be98-47496985a035-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:02:46.789927 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.789899 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aff37a45-e0f2-4cf1-be98-47496985a035-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aff37a45-e0f2-4cf1-be98-47496985a035" (UID: "aff37a45-e0f2-4cf1-be98-47496985a035"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 17:02:46.790021 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.789978 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aff37a45-e0f2-4cf1-be98-47496985a035" (UID: "aff37a45-e0f2-4cf1-be98-47496985a035"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:02:46.790021 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.790010 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-kube-api-access-pggml" (OuterVolumeSpecName: "kube-api-access-pggml") pod "aff37a45-e0f2-4cf1-be98-47496985a035" (UID: "aff37a45-e0f2-4cf1-be98-47496985a035"). InnerVolumeSpecName "kube-api-access-pggml". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:02:46.790170 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.790154 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-istio-token" (OuterVolumeSpecName: "istio-token") pod "aff37a45-e0f2-4cf1-be98-47496985a035" (UID: "aff37a45-e0f2-4cf1-be98-47496985a035"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:02:46.888857 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.888830 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff37a45-e0f2-4cf1-be98-47496985a035-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:02:46.888857 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.888853 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:02:46.888857 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.888862 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pggml\" (UniqueName: \"kubernetes.io/projected/aff37a45-e0f2-4cf1-be98-47496985a035-kube-api-access-pggml\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:02:46.889078 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:46.888872 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff37a45-e0f2-4cf1-be98-47496985a035-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:02:47.514911 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.514874 2582 generic.go:356] "Generic (PLEG): container finished" podID="aff37a45-e0f2-4cf1-be98-47496985a035" containerID="0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd" exitCode=0 Aug 03 17:02:47.515099 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.514923 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" Aug 03 17:02:47.515099 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.514958 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" event={"ID":"aff37a45-e0f2-4cf1-be98-47496985a035","Type":"ContainerDied","Data":"0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd"} Aug 03 17:02:47.515099 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.515013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p" event={"ID":"aff37a45-e0f2-4cf1-be98-47496985a035","Type":"ContainerDied","Data":"2b64fb413cfb2b4c9750a140225793f9df5174168c676831c15024b00742da09"} Aug 03 17:02:47.515099 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.515036 2582 scope.go:117] "RemoveContainer" containerID="0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd" Aug 03 17:02:47.524916 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.524898 2582 scope.go:117] "RemoveContainer" containerID="f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a" Aug 03 17:02:47.533190 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.533175 2582 scope.go:117] "RemoveContainer" containerID="0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd" Aug 03 17:02:47.533456 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:02:47.533438 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd\": container with ID starting with 0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd not found: ID does not exist" containerID="0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd" Aug 03 17:02:47.533507 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.533464 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd"} err="failed to get container status \"0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd\": rpc error: code = NotFound desc = could not find container \"0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd\": container with ID starting with 0be0f7f39979d35ea97dcab3b640ee35b247993eedffb19f5e103d8a373e51dd not found: ID does not exist" Aug 03 17:02:47.533507 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.533481 2582 scope.go:117] "RemoveContainer" containerID="f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a" Aug 03 17:02:47.533722 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:02:47.533702 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a\": container with ID starting with f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a not found: ID does not exist" containerID="f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a" Aug 03 17:02:47.533783 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.533733 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a"} err="failed to get container status \"f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a\": rpc error: code = NotFound desc = could not find container \"f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a\": container with ID starting with f55efb3097998e7643e4df455307327c9c38bb6e00510a630712e3fdab55716a not found: ID does not exist" Aug 03 17:02:47.541261 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.541240 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p"] Aug 03 17:02:47.545135 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.545114 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-0e4b78-openshift-default-7c5894cfdb-tzc2p"] Aug 03 17:02:47.994806 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:47.994727 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" path="/var/lib/kubelet/pods/aff37a45-e0f2-4cf1-be98-47496985a035/volumes" Aug 03 17:02:59.558890 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:59.558856 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg"] Aug 03 17:02:59.559324 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:02:59.559118 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" containerID="cri-o://f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053" gracePeriod=30 Aug 03 17:03:04.803514 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.803491 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 17:03:04.958645 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958534 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423be317-b793-47e8-8ae8-ba18dddf6223-istiod-ca-cert\") pod \"423be317-b793-47e8-8ae8-ba18dddf6223\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " Aug 03 17:03:04.958645 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958572 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423be317-b793-47e8-8ae8-ba18dddf6223-istio-podinfo\") pod \"423be317-b793-47e8-8ae8-ba18dddf6223\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " Aug 03 17:03:04.958645 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958593 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-socket\") pod \"423be317-b793-47e8-8ae8-ba18dddf6223\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " Aug 03 17:03:04.958645 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958633 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-data\") pod \"423be317-b793-47e8-8ae8-ba18dddf6223\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " Aug 03 17:03:04.958973 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958697 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-istio-token\") pod \"423be317-b793-47e8-8ae8-ba18dddf6223\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " Aug 03 17:03:04.958973 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958742 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-credential-socket\") pod \"423be317-b793-47e8-8ae8-ba18dddf6223\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " Aug 03 17:03:04.958973 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958773 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfvtb\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-kube-api-access-cfvtb\") pod \"423be317-b793-47e8-8ae8-ba18dddf6223\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " Aug 03 17:03:04.958973 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958816 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-envoy\") pod \"423be317-b793-47e8-8ae8-ba18dddf6223\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " Aug 03 17:03:04.958973 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958852 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-certs\") pod \"423be317-b793-47e8-8ae8-ba18dddf6223\" (UID: \"423be317-b793-47e8-8ae8-ba18dddf6223\") " Aug 03 17:03:04.958973 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958917 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "423be317-b793-47e8-8ae8-ba18dddf6223" (UID: "423be317-b793-47e8-8ae8-ba18dddf6223"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:04.958973 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.958928 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-data" (OuterVolumeSpecName: "istio-data") pod "423be317-b793-47e8-8ae8-ba18dddf6223" (UID: "423be317-b793-47e8-8ae8-ba18dddf6223"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:04.959336 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.959019 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/423be317-b793-47e8-8ae8-ba18dddf6223-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "423be317-b793-47e8-8ae8-ba18dddf6223" (UID: "423be317-b793-47e8-8ae8-ba18dddf6223"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 17:03:04.959336 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.959075 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "423be317-b793-47e8-8ae8-ba18dddf6223" (UID: "423be317-b793-47e8-8ae8-ba18dddf6223"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:04.959336 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.959250 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/423be317-b793-47e8-8ae8-ba18dddf6223-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:04.959336 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.959279 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:04.959336 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.959289 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:04.959336 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.959297 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:04.959336 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.959305 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "423be317-b793-47e8-8ae8-ba18dddf6223" (UID: "423be317-b793-47e8-8ae8-ba18dddf6223"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:04.961121 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.961092 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-kube-api-access-cfvtb" (OuterVolumeSpecName: "kube-api-access-cfvtb") pod "423be317-b793-47e8-8ae8-ba18dddf6223" (UID: "423be317-b793-47e8-8ae8-ba18dddf6223"). InnerVolumeSpecName "kube-api-access-cfvtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:03:04.961230 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.961100 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-istio-token" (OuterVolumeSpecName: "istio-token") pod "423be317-b793-47e8-8ae8-ba18dddf6223" (UID: "423be317-b793-47e8-8ae8-ba18dddf6223"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:03:04.961350 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.961328 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "423be317-b793-47e8-8ae8-ba18dddf6223" (UID: "423be317-b793-47e8-8ae8-ba18dddf6223"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:04.961427 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:04.961409 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/423be317-b793-47e8-8ae8-ba18dddf6223-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "423be317-b793-47e8-8ae8-ba18dddf6223" (UID: "423be317-b793-47e8-8ae8-ba18dddf6223"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 17:03:05.060207 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.060177 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/423be317-b793-47e8-8ae8-ba18dddf6223-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:05.060207 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.060202 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:05.060207 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.060211 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfvtb\" (UniqueName: \"kubernetes.io/projected/423be317-b793-47e8-8ae8-ba18dddf6223-kube-api-access-cfvtb\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:05.060435 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.060221 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:05.060435 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.060231 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/423be317-b793-47e8-8ae8-ba18dddf6223-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:05.620143 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.620112 2582 generic.go:356] "Generic (PLEG): container finished" podID="423be317-b793-47e8-8ae8-ba18dddf6223" containerID="f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053" exitCode=0 Aug 03 17:03:05.620326 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.620158 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" Aug 03 17:03:05.620326 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.620160 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" event={"ID":"423be317-b793-47e8-8ae8-ba18dddf6223","Type":"ContainerDied","Data":"f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053"} Aug 03 17:03:05.620326 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.620284 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg" event={"ID":"423be317-b793-47e8-8ae8-ba18dddf6223","Type":"ContainerDied","Data":"b0cb0531c0d15ad15c0ab81a6c4aeb856a124690ce3de3b1ec53851f675abfb6"} Aug 03 17:03:05.620326 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.620311 2582 scope.go:117] "RemoveContainer" containerID="f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053" Aug 03 17:03:05.630197 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.630182 2582 scope.go:117] "RemoveContainer" containerID="ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f" Aug 03 17:03:05.638033 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.638015 2582 scope.go:117] "RemoveContainer" containerID="f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053" Aug 03 17:03:05.638265 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:03:05.638249 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053\": container with ID starting with f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053 not found: ID does not exist" containerID="f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053" Aug 03 17:03:05.638328 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.638272 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053"} err="failed to get container status \"f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053\": rpc error: code = NotFound desc = could not find container \"f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053\": container with ID starting with f777dbde5dc4bbc6a9a143e6cfae26c1c71f7959817603849efb0d161c784053 not found: ID does not exist" Aug 03 17:03:05.638328 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.638287 2582 scope.go:117] "RemoveContainer" containerID="ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f" Aug 03 17:03:05.638507 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:03:05.638492 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f\": container with ID starting with ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f not found: ID does not exist" containerID="ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f" Aug 03 17:03:05.638554 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.638511 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f"} err="failed to get container status \"ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f\": rpc error: code = NotFound desc = could not find container \"ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f\": container with ID starting with ff7d329083feef2c038da1a04973c0661c718be390b6d96f6e01e168e39db10f not found: ID does not exist" Aug 03 17:03:05.644390 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.644370 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg"] Aug 03 17:03:05.649674 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.649655 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-cf8ed9-openshift-default-64bf9445cd-fqjvg"] Aug 03 17:03:05.995495 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:05.995416 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" path="/var/lib/kubelet/pods/423be317-b793-47e8-8ae8-ba18dddf6223/volumes" Aug 03 17:03:18.157799 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:18.157765 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95"] Aug 03 17:03:18.158362 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:18.157978 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" containerID="cri-o://901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd" gracePeriod=30 Aug 03 17:03:21.522008 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:21.521970 2582 patch_prober.go:28] interesting pod/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 03 17:03:21.522387 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:21.522036 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 03 17:03:23.411060 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.411034 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 17:03:23.516850 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.516818 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-credential-socket\") pod \"029af2b3-97b9-42b4-88fb-652e3fe40960\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " Aug 03 17:03:23.517024 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.516864 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-socket\") pod \"029af2b3-97b9-42b4-88fb-652e3fe40960\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " Aug 03 17:03:23.517024 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.516894 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-certs\") pod \"029af2b3-97b9-42b4-88fb-652e3fe40960\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " Aug 03 17:03:23.517024 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.516910 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/029af2b3-97b9-42b4-88fb-652e3fe40960-istiod-ca-cert\") pod \"029af2b3-97b9-42b4-88fb-652e3fe40960\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " Aug 03 17:03:23.517024 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.516937 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn7d7\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-kube-api-access-nn7d7\") pod \"029af2b3-97b9-42b4-88fb-652e3fe40960\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " Aug 03 17:03:23.517024 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.516954 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-token\") pod \"029af2b3-97b9-42b4-88fb-652e3fe40960\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " Aug 03 17:03:23.517271 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517049 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-data\") pod \"029af2b3-97b9-42b4-88fb-652e3fe40960\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " Aug 03 17:03:23.517271 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517086 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-envoy\") pod \"029af2b3-97b9-42b4-88fb-652e3fe40960\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " Aug 03 17:03:23.517271 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517108 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-podinfo\") pod \"029af2b3-97b9-42b4-88fb-652e3fe40960\" (UID: \"029af2b3-97b9-42b4-88fb-652e3fe40960\") " Aug 03 17:03:23.517271 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517143 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "029af2b3-97b9-42b4-88fb-652e3fe40960" (UID: "029af2b3-97b9-42b4-88fb-652e3fe40960"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:23.517271 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517197 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "029af2b3-97b9-42b4-88fb-652e3fe40960" (UID: "029af2b3-97b9-42b4-88fb-652e3fe40960"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:23.517271 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517238 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "029af2b3-97b9-42b4-88fb-652e3fe40960" (UID: "029af2b3-97b9-42b4-88fb-652e3fe40960"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:23.517607 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517316 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-data" (OuterVolumeSpecName: "istio-data") pod "029af2b3-97b9-42b4-88fb-652e3fe40960" (UID: "029af2b3-97b9-42b4-88fb-652e3fe40960"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:23.517607 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517347 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/029af2b3-97b9-42b4-88fb-652e3fe40960-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "029af2b3-97b9-42b4-88fb-652e3fe40960" (UID: "029af2b3-97b9-42b4-88fb-652e3fe40960"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 17:03:23.517607 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517514 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:23.517607 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517535 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:23.517607 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517550 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:23.517607 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517562 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:23.517607 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.517573 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/029af2b3-97b9-42b4-88fb-652e3fe40960-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:23.519275 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.519242 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "029af2b3-97b9-42b4-88fb-652e3fe40960" (UID: "029af2b3-97b9-42b4-88fb-652e3fe40960"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 17:03:23.519275 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.519262 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-token" (OuterVolumeSpecName: "istio-token") pod "029af2b3-97b9-42b4-88fb-652e3fe40960" (UID: "029af2b3-97b9-42b4-88fb-652e3fe40960"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:03:23.519435 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.519304 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-kube-api-access-nn7d7" (OuterVolumeSpecName: "kube-api-access-nn7d7") pod "029af2b3-97b9-42b4-88fb-652e3fe40960" (UID: "029af2b3-97b9-42b4-88fb-652e3fe40960"). InnerVolumeSpecName "kube-api-access-nn7d7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:03:23.519435 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.519368 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "029af2b3-97b9-42b4-88fb-652e3fe40960" (UID: "029af2b3-97b9-42b4-88fb-652e3fe40960"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:23.618833 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.618800 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:23.618833 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.618825 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:23.618833 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.618836 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn7d7\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-kube-api-access-nn7d7\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:23.619058 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.618846 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/029af2b3-97b9-42b4-88fb-652e3fe40960-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:23.728417 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.728382 2582 generic.go:356] "Generic (PLEG): container finished" podID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerID="901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd" exitCode=0 Aug 03 17:03:23.728592 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.728435 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" Aug 03 17:03:23.728592 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.728465 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" event={"ID":"029af2b3-97b9-42b4-88fb-652e3fe40960","Type":"ContainerDied","Data":"901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd"} Aug 03 17:03:23.728592 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.728524 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95" event={"ID":"029af2b3-97b9-42b4-88fb-652e3fe40960","Type":"ContainerDied","Data":"6df78bb18054d86caa4c4f85db0543981c4c959eedda4c3a9cdbd4c3ca5e1c93"} Aug 03 17:03:23.728592 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.728559 2582 scope.go:117] "RemoveContainer" containerID="901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd" Aug 03 17:03:23.738031 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.738014 2582 scope.go:117] "RemoveContainer" containerID="e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17" Aug 03 17:03:23.746350 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.746334 2582 scope.go:117] "RemoveContainer" containerID="901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd" Aug 03 17:03:23.746593 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:03:23.746574 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd\": container with ID starting with 901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd not found: ID does not exist" containerID="901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd" Aug 03 17:03:23.746700 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.746606 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd"} err="failed to get container status \"901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd\": rpc error: code = NotFound desc = could not find container \"901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd\": container with ID starting with 901bd2e7ec6192e8b310532328d1618e0c295e3f2dab9b07405ffdea23cf37cd not found: ID does not exist" Aug 03 17:03:23.746700 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.746643 2582 scope.go:117] "RemoveContainer" containerID="e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17" Aug 03 17:03:23.746892 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:03:23.746872 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17\": container with ID starting with e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17 not found: ID does not exist" containerID="e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17" Aug 03 17:03:23.746940 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.746898 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17"} err="failed to get container status \"e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17\": rpc error: code = NotFound desc = could not find container \"e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17\": container with ID starting with e7cab7462e6d66e7d707b3d44c021d640f163b8f2289fe82729d04e33a771a17 not found: ID does not exist" Aug 03 17:03:23.751867 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.751843 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95"] Aug 03 17:03:23.756231 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.756209 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-0fae8e-openshift-default-58b7f645d4-m2f95"] Aug 03 17:03:23.995914 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:23.995821 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" path="/var/lib/kubelet/pods/029af2b3-97b9-42b4-88fb-652e3fe40960/volumes" Aug 03 17:03:36.187520 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:36.187484 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm"] Aug 03 17:03:36.187983 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:36.187739 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" containerID="cri-o://2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f" gracePeriod=30 Aug 03 17:03:37.019685 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:37.019640 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dc447499-lwc8r_07341eab-9128-49cf-948a-b14235ad42da/authorino/0.log" Aug 03 17:03:41.447182 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.447158 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 17:03:41.586344 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586312 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-envoy\") pod \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " Aug 03 17:03:41.586344 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586346 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-podinfo\") pod \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " Aug 03 17:03:41.586559 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586386 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-credential-socket\") pod \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " Aug 03 17:03:41.586597 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586561 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istiod-ca-cert\") pod \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " Aug 03 17:03:41.586673 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586603 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-data\") pod \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " Aug 03 17:03:41.586730 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586691 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-token\") pod \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " Aug 03 17:03:41.586730 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586702 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" (UID: "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:41.586730 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586724 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csxb2\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-kube-api-access-csxb2\") pod \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " Aug 03 17:03:41.586880 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586759 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-certs\") pod \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " Aug 03 17:03:41.586880 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.586788 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-socket\") pod \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\" (UID: \"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82\") " Aug 03 17:03:41.587223 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.587182 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" (UID: "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 03 17:03:41.587414 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.587340 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" (UID: "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:41.587550 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.587409 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" (UID: "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:41.587758 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.587707 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" (UID: "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:41.588575 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.588549 2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-credential-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:41.588714 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.588586 2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istiod-ca-cert\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:41.588714 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.588601 2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-data\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:41.588714 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.588613 2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-certs\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:41.588714 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.588647 2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-workload-socket\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:41.589581 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.589552 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" (UID: "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 03 17:03:41.589887 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.589865 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" (UID: "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 03 17:03:41.591661 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.590400 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" (UID: "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:03:41.596493 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.596466 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-kube-api-access-csxb2" (OuterVolumeSpecName: "kube-api-access-csxb2") pod "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" (UID: "aa35e65d-d3ab-4aa0-90d3-18c9825c2c82"). InnerVolumeSpecName "kube-api-access-csxb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 03 17:03:41.689477 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.689447 2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-token\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:41.689477 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.689475 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csxb2\" (UniqueName: \"kubernetes.io/projected/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-kube-api-access-csxb2\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:41.689683 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.689486 2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-envoy\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:41.689683 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.689495 2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82-istio-podinfo\") on node \"ip-10-0-142-140.ec2.internal\" DevicePath \"\"" Aug 03 17:03:41.830281 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.830242 2582 generic.go:356] "Generic (PLEG): container finished" podID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerID="2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f" exitCode=0 Aug 03 17:03:41.830446 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.830310 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" Aug 03 17:03:41.830446 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.830321 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" event={"ID":"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82","Type":"ContainerDied","Data":"2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f"} Aug 03 17:03:41.830446 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.830397 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm" event={"ID":"aa35e65d-d3ab-4aa0-90d3-18c9825c2c82","Type":"ContainerDied","Data":"15716f1b3040014ad035f145a497eddaa42efc851db5ef4a4da2bb0eb5746ef0"} Aug 03 17:03:41.830446 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.830420 2582 scope.go:117] "RemoveContainer" containerID="2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f" Aug 03 17:03:41.839934 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.839915 2582 scope.go:117] "RemoveContainer" containerID="41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983" Aug 03 17:03:41.847931 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.847914 2582 scope.go:117] "RemoveContainer" containerID="2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f" Aug 03 17:03:41.848186 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:03:41.848163 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f\": container with ID starting with 2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f not found: ID does not exist" containerID="2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f" Aug 03 17:03:41.848252 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.848193 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f"} err="failed to get container status \"2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f\": rpc error: code = NotFound desc = could not find container \"2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f\": container with ID starting with 2e85643c223ee3acc9b7e0ae1f6ff405f76c03e4cb52f63c0ed53a88d93e044f not found: ID does not exist" Aug 03 17:03:41.848252 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.848208 2582 scope.go:117] "RemoveContainer" containerID="41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983" Aug 03 17:03:41.848429 ip-10-0-142-140 kubenswrapper[2582]: E0803 17:03:41.848411 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983\": container with ID starting with 41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983 not found: ID does not exist" containerID="41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983" Aug 03 17:03:41.848490 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.848439 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983"} err="failed to get container status \"41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983\": rpc error: code = NotFound desc = could not find container \"41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983\": container with ID starting with 41fab50d14d6f2d3f7cb8d2544b68a369cc07be9fd94ca3b1ef194fb2bebe983 not found: ID does not exist" Aug 03 17:03:41.854381 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.854361 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm"] Aug 03 17:03:41.859098 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.859079 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-7055f0-openshift-default-7f887b656d-xhtpm"] Aug 03 17:03:41.994918 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:41.994885 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" path="/var/lib/kubelet/pods/aa35e65d-d3ab-4aa0-90d3-18c9825c2c82/volumes" Aug 03 17:03:44.295259 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:44.295233 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6xmkz_6335474c-f23a-45a3-a678-cd809b190022/server/0.log" Aug 03 17:03:44.783963 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:44.783936 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-c2wvc_618e12cf-4fbb-4dfb-ad05-2095f55819fc/copy-manifests/0.log" Aug 03 17:03:44.791025 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:44.791001 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-c2wvc_618e12cf-4fbb-4dfb-ad05-2095f55819fc/manager/0.log" Aug 03 17:03:46.921016 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:46.920989 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dc447499-lwc8r_07341eab-9128-49cf-948a-b14235ad42da/authorino/0.log" Aug 03 17:03:47.059530 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:47.059494 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-x8fwc_0f459b6a-8dfc-4626-9ee2-d5d025bc3a37/manager/0.log" Aug 03 17:03:47.208839 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:47.208760 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-r8g86_55013419-ac83-48b8-a590-93275df3b798/manager/0.log" Aug 03 17:03:47.492741 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:47.492667 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-mg52d_e404b6bd-15b3-4269-921b-02e6dbbda707/manager/0.log" Aug 03 17:03:48.209571 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:48.209542 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg_8758d0d7-c58c-4266-b081-d8d598272127/istio-proxy/0.log" Aug 03 17:03:48.801126 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:48.801094 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-4htkn_8db2abdc-9a5a-49a2-9df8-909dacd7cf61/istio-proxy/1.log" Aug 03 17:03:49.834513 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:49.834486 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-j2sr2_103c8f44-468a-469b-a846-127395769464/main/0.log" Aug 03 17:03:50.127971 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:50.127875 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fhfm2z_47d84110-6dcc-421c-b1df-a1c79c825ebe/main/0.log" Aug 03 17:03:57.580040 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580014 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xpjhq/must-gather-mhzr6"] Aug 03 17:03:57.580405 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580361 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" Aug 03 17:03:57.580405 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580371 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" Aug 03 17:03:57.580405 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580379 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" Aug 03 17:03:57.580405 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580385 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" Aug 03 17:03:57.580405 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580393 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" Aug 03 17:03:57.580405 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580398 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580411 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580416 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580424 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580429 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580440 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580445 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580451 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580456 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580462 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580467 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580521 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580528 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580534 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580540 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580546 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580552 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580558 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa35e65d-d3ab-4aa0-90d3-18c9825c2c82" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580563 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" Aug 03 17:03:57.580595 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580568 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff37a45-e0f2-4cf1-be98-47496985a035" containerName="istio-proxy" Aug 03 17:03:57.581338 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580634 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" Aug 03 17:03:57.581338 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580640 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="423be317-b793-47e8-8ae8-ba18dddf6223" containerName="istio-proxy" Aug 03 17:03:57.581338 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580646 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" Aug 03 17:03:57.581338 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580651 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="029af2b3-97b9-42b4-88fb-652e3fe40960" containerName="istio-proxy" Aug 03 17:03:57.581338 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.580724 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aea84f1b-0f91-4e29-9a58-cac3e316b88a" containerName="istio-proxy" Aug 03 17:03:57.585562 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.585543 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpjhq/must-gather-mhzr6" Aug 03 17:03:57.591428 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.591408 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpjhq\"/\"openshift-service-ca.crt\"" Aug 03 17:03:57.592736 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.592715 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xpjhq\"/\"default-dockercfg-dl5pw\"" Aug 03 17:03:57.592818 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.592725 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpjhq\"/\"kube-root-ca.crt\"" Aug 03 17:03:57.597169 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.597135 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpjhq/must-gather-mhzr6"] Aug 03 17:03:57.725876 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.725855 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm4d8\" (UniqueName: \"kubernetes.io/projected/2eb8a6ae-1297-4653-908f-a206c90bead2-kube-api-access-tm4d8\") pod \"must-gather-mhzr6\" (UID: \"2eb8a6ae-1297-4653-908f-a206c90bead2\") " pod="openshift-must-gather-xpjhq/must-gather-mhzr6" Aug 03 17:03:57.726004 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.725898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2eb8a6ae-1297-4653-908f-a206c90bead2-must-gather-output\") pod \"must-gather-mhzr6\" (UID: \"2eb8a6ae-1297-4653-908f-a206c90bead2\") " pod="openshift-must-gather-xpjhq/must-gather-mhzr6" Aug 03 17:03:57.827153 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.827129 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm4d8\" (UniqueName: \"kubernetes.io/projected/2eb8a6ae-1297-4653-908f-a206c90bead2-kube-api-access-tm4d8\") pod \"must-gather-mhzr6\" (UID: \"2eb8a6ae-1297-4653-908f-a206c90bead2\") " pod="openshift-must-gather-xpjhq/must-gather-mhzr6" Aug 03 17:03:57.827267 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.827173 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2eb8a6ae-1297-4653-908f-a206c90bead2-must-gather-output\") pod \"must-gather-mhzr6\" (UID: \"2eb8a6ae-1297-4653-908f-a206c90bead2\") " pod="openshift-must-gather-xpjhq/must-gather-mhzr6" Aug 03 17:03:57.827458 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.827443 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2eb8a6ae-1297-4653-908f-a206c90bead2-must-gather-output\") pod \"must-gather-mhzr6\" (UID: \"2eb8a6ae-1297-4653-908f-a206c90bead2\") " pod="openshift-must-gather-xpjhq/must-gather-mhzr6" Aug 03 17:03:57.841302 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.841239 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm4d8\" (UniqueName: \"kubernetes.io/projected/2eb8a6ae-1297-4653-908f-a206c90bead2-kube-api-access-tm4d8\") pod \"must-gather-mhzr6\" (UID: \"2eb8a6ae-1297-4653-908f-a206c90bead2\") " pod="openshift-must-gather-xpjhq/must-gather-mhzr6" Aug 03 17:03:57.895180 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:57.895155 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpjhq/must-gather-mhzr6" Aug 03 17:03:58.042543 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:58.042517 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpjhq/must-gather-mhzr6"] Aug 03 17:03:58.043451 ip-10-0-142-140 kubenswrapper[2582]: W0803 17:03:58.043426 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2eb8a6ae_1297_4653_908f_a206c90bead2.slice/crio-1a001f8c8d718fa8674c19f96fe5784cf4696ea67dbbd31abc92ca5fe556fb0d WatchSource:0}: Error finding container 1a001f8c8d718fa8674c19f96fe5784cf4696ea67dbbd31abc92ca5fe556fb0d: Status 404 returned error can't find the container with id 1a001f8c8d718fa8674c19f96fe5784cf4696ea67dbbd31abc92ca5fe556fb0d Aug 03 17:03:58.045345 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:58.045329 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 03 17:03:58.928038 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:58.928010 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpjhq/must-gather-mhzr6" event={"ID":"2eb8a6ae-1297-4653-908f-a206c90bead2","Type":"ContainerStarted","Data":"1a001f8c8d718fa8674c19f96fe5784cf4696ea67dbbd31abc92ca5fe556fb0d"} Aug 03 17:03:59.936853 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:59.936818 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpjhq/must-gather-mhzr6" event={"ID":"2eb8a6ae-1297-4653-908f-a206c90bead2","Type":"ContainerStarted","Data":"ed0cdd534935f24039caa38e6ce4c49876816fc78c528c8b9614962a4ae74047"} Aug 03 17:03:59.937291 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:59.936865 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpjhq/must-gather-mhzr6" event={"ID":"2eb8a6ae-1297-4653-908f-a206c90bead2","Type":"ContainerStarted","Data":"0db9f1b0b035a7840757108260cb3bbd15ceb327afc0f669eccd207e0cbb6fed"} Aug 03 17:03:59.957469 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:03:59.957407 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xpjhq/must-gather-mhzr6" podStartSLOduration=2.1297842080000002 podStartE2EDuration="2.957384412s" totalImagesPullingTime="827.600204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-03 17:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:03:59.954448738 +0000 UTC m=+3578.512531324" watchObservedRunningTime="2026-08-03 17:03:59.957384412 +0000 UTC m=+3578.515466998" Aug 03 17:04:00.569194 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:00.569147 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rq67m_16baf6d7-7c56-4d29-82bd-36c91c330974/global-pull-secret-syncer/0.log" Aug 03 17:04:00.745436 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:00.745408 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n9756_3a667877-9466-493b-beb0-fb6c194c11c9/konnectivity-agent/0.log" Aug 03 17:04:00.817910 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:00.817883 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-140.ec2.internal_6e7f1ea143023130aaaa7a2dabe0813a/haproxy/0.log" Aug 03 17:04:05.435554 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:05.435400 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6dc447499-lwc8r_07341eab-9128-49cf-948a-b14235ad42da/authorino/0.log" Aug 03 17:04:05.472801 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:05.472769 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-x8fwc_0f459b6a-8dfc-4626-9ee2-d5d025bc3a37/manager/0.log" Aug 03 17:04:05.508986 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:05.508962 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-r8g86_55013419-ac83-48b8-a590-93275df3b798/manager/0.log" Aug 03 17:04:05.677642 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:05.673629 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-mg52d_e404b6bd-15b3-4269-921b-02e6dbbda707/manager/0.log" Aug 03 17:04:07.229878 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.229844 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4860400e-2aaa-4232-b1e7-219eb243093f/alertmanager/0.log" Aug 03 17:04:07.257908 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.257874 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4860400e-2aaa-4232-b1e7-219eb243093f/config-reloader/0.log" Aug 03 17:04:07.295794 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.295754 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4860400e-2aaa-4232-b1e7-219eb243093f/kube-rbac-proxy-web/0.log" Aug 03 17:04:07.335090 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.334841 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4860400e-2aaa-4232-b1e7-219eb243093f/kube-rbac-proxy/0.log" Aug 03 17:04:07.367426 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.367367 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4860400e-2aaa-4232-b1e7-219eb243093f/kube-rbac-proxy-metric/0.log" Aug 03 17:04:07.393236 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.393202 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4860400e-2aaa-4232-b1e7-219eb243093f/prom-label-proxy/0.log" Aug 03 17:04:07.420039 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.420015 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4860400e-2aaa-4232-b1e7-219eb243093f/init-config-reloader/0.log" Aug 03 17:04:07.552830 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.552800 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-5j8r7_65de89b0-fa4c-4b10-914e-a2a62557317e/kube-state-metrics/0.log" Aug 03 17:04:07.584812 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.584723 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-5j8r7_65de89b0-fa4c-4b10-914e-a2a62557317e/kube-rbac-proxy-main/0.log" Aug 03 17:04:07.614040 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.613996 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-6bd9664c4-5j8r7_65de89b0-fa4c-4b10-914e-a2a62557317e/kube-rbac-proxy-self/0.log" Aug 03 17:04:07.846939 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.846857 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p5hcc_17618d89-6184-4883-a2a1-91c1b35c3625/node-exporter/0.log" Aug 03 17:04:07.877324 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.877284 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p5hcc_17618d89-6184-4883-a2a1-91c1b35c3625/kube-rbac-proxy/0.log" Aug 03 17:04:07.903581 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:07.903541 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p5hcc_17618d89-6184-4883-a2a1-91c1b35c3625/init-textfile/0.log" Aug 03 17:04:08.046042 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.046010 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-8w9zl_a3b2df13-0345-49e4-a33b-2c367fcdce45/kube-rbac-proxy-main/0.log" Aug 03 17:04:08.072574 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.072548 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-8w9zl_a3b2df13-0345-49e4-a33b-2c367fcdce45/kube-rbac-proxy-self/0.log" Aug 03 17:04:08.104473 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.104399 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6bb4d7b4dd-8w9zl_a3b2df13-0345-49e4-a33b-2c367fcdce45/openshift-state-metrics/0.log" Aug 03 17:04:08.420274 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.420193 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-66685ff6d4-z4w99_d6f748fd-5da6-4f68-9786-0975f6fabdbb/prometheus-operator-admission-webhook/0.log" Aug 03 17:04:08.456279 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.456245 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cf644d898-7qqpr_318cc044-cddf-40e5-8285-8bf12b11949d/telemeter-client/0.log" Aug 03 17:04:08.480402 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.480372 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cf644d898-7qqpr_318cc044-cddf-40e5-8285-8bf12b11949d/reload/0.log" Aug 03 17:04:08.509005 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.508970 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cf644d898-7qqpr_318cc044-cddf-40e5-8285-8bf12b11949d/kube-rbac-proxy/0.log" Aug 03 17:04:08.552846 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.552822 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668d95d56c-862fz_bfa9e619-6beb-4f50-94ce-bb403aa164b6/thanos-query/0.log" Aug 03 17:04:08.578088 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.578047 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668d95d56c-862fz_bfa9e619-6beb-4f50-94ce-bb403aa164b6/kube-rbac-proxy-web/0.log" Aug 03 17:04:08.603897 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.603872 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668d95d56c-862fz_bfa9e619-6beb-4f50-94ce-bb403aa164b6/kube-rbac-proxy/0.log" Aug 03 17:04:08.628012 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.627985 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668d95d56c-862fz_bfa9e619-6beb-4f50-94ce-bb403aa164b6/prom-label-proxy/0.log" Aug 03 17:04:08.655499 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.655475 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668d95d56c-862fz_bfa9e619-6beb-4f50-94ce-bb403aa164b6/kube-rbac-proxy-rules/0.log" Aug 03 17:04:08.684083 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:08.684011 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668d95d56c-862fz_bfa9e619-6beb-4f50-94ce-bb403aa164b6/kube-rbac-proxy-metrics/0.log" Aug 03 17:04:10.290243 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.290203 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4"] Aug 03 17:04:10.298291 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.298262 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.308398 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.308373 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4"] Aug 03 17:04:10.393830 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.393459 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-sys\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.393830 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.393517 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-lib-modules\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.393830 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.393545 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-podres\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.393830 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.393608 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbl4r\" (UniqueName: \"kubernetes.io/projected/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-kube-api-access-fbl4r\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.393830 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.393683 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-proc\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.494517 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.494478 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-sys\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.494739 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.494549 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-lib-modules\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.494739 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.494575 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-podres\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.494739 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.494668 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbl4r\" (UniqueName: \"kubernetes.io/projected/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-kube-api-access-fbl4r\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.494739 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.494723 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-proc\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.494955 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.494735 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-lib-modules\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.494955 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.494796 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-sys\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.494955 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.494840 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-proc\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.495109 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.494735 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-podres\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.507501 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.507475 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbl4r\" (UniqueName: \"kubernetes.io/projected/4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58-kube-api-access-fbl4r\") pod \"perf-node-gather-daemonset-c9hv4\" (UID: \"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58\") " pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.617783 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.617479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:10.799601 ip-10-0-142-140 kubenswrapper[2582]: W0803 17:04:10.799569 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4fa6f1e7_7a31_4c51_b9e4_dcf0e05f2d58.slice/crio-7784f2e8508f9adc4a8d9ca773dc277fd373331cbe275d0b32f527b64a3dba6a WatchSource:0}: Error finding container 7784f2e8508f9adc4a8d9ca773dc277fd373331cbe275d0b32f527b64a3dba6a: Status 404 returned error can't find the container with id 7784f2e8508f9adc4a8d9ca773dc277fd373331cbe275d0b32f527b64a3dba6a Aug 03 17:04:10.800059 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:10.800036 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4"] Aug 03 17:04:11.039281 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:11.039230 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" event={"ID":"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58","Type":"ContainerStarted","Data":"0197b2ca4d214398d3c333cd2e10401b933b170b015bee65083b9b6503a04943"} Aug 03 17:04:11.039477 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:11.039352 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" event={"ID":"4fa6f1e7-7a31-4c51-b9e4-dcf0e05f2d58","Type":"ContainerStarted","Data":"7784f2e8508f9adc4a8d9ca773dc277fd373331cbe275d0b32f527b64a3dba6a"} Aug 03 17:04:11.040202 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:11.039590 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:11.063203 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:11.063122 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" podStartSLOduration=1.063104569 podStartE2EDuration="1.063104569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-03 17:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-03 17:04:11.058687465 +0000 UTC m=+3589.616770050" watchObservedRunningTime="2026-08-03 17:04:11.063104569 +0000 UTC m=+3589.621187158" Aug 03 17:04:13.165328 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:13.165299 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xwhgl_bc59383f-9acf-44f5-85a0-fe7eb7189356/dns/0.log" Aug 03 17:04:13.213658 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:13.213609 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xwhgl_bc59383f-9acf-44f5-85a0-fe7eb7189356/kube-rbac-proxy/0.log" Aug 03 17:04:13.290482 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:13.290456 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8g4rz_b9a4751f-3a31-448f-a115-8d487b785137/dns-node-resolver/0.log" Aug 03 17:04:13.949736 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:13.949703 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2vkng_c443b634-9b93-4578-b47b-00a9cb4be95b/node-ca/0.log" Aug 03 17:04:15.091342 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:15.091317 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg_8758d0d7-c58c-4266-b081-d8d598272127/istio-proxy/0.log" Aug 03 17:04:15.502717 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:15.502647 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-4htkn_8db2abdc-9a5a-49a2-9df8-909dacd7cf61/istio-proxy/0.log" Aug 03 17:04:15.510553 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:15.510531 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-4htkn_8db2abdc-9a5a-49a2-9df8-909dacd7cf61/istio-proxy/1.log" Aug 03 17:04:16.461694 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:16.461653 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7hj45_1024e841-024a-4af9-a182-c08ab00e2bf8/serve-healthcheck-canary/0.log" Aug 03 17:04:17.055228 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:17.055199 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xpjhq/perf-node-gather-daemonset-c9hv4" Aug 03 17:04:17.089204 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:17.089179 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4xtzr_b4167a63-fa6c-427a-8be0-51fb4be4540d/kube-rbac-proxy/0.log" Aug 03 17:04:17.129460 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:17.129429 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4xtzr_b4167a63-fa6c-427a-8be0-51fb4be4540d/exporter/0.log" Aug 03 17:04:17.158006 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:17.157979 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4xtzr_b4167a63-fa6c-427a-8be0-51fb4be4540d/extractor/0.log" Aug 03 17:04:20.151066 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:20.150990 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6xmkz_6335474c-f23a-45a3-a678-cd809b190022/server/0.log" Aug 03 17:04:20.314199 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:20.314171 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-c2wvc_618e12cf-4fbb-4dfb-ad05-2095f55819fc/manager/0.log" Aug 03 17:04:20.341887 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:20.341863 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4d846dc9-c2wvc_618e12cf-4fbb-4dfb-ad05-2095f55819fc/copy-manifests/0.log" Aug 03 17:04:20.529741 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:20.529684 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg_8758d0d7-c58c-4266-b081-d8d598272127/istio-proxy/0.log" Aug 03 17:04:20.896245 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:20.896163 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-4htkn_8db2abdc-9a5a-49a2-9df8-909dacd7cf61/istio-proxy/0.log" Aug 03 17:04:20.903714 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:20.903693 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-4htkn_8db2abdc-9a5a-49a2-9df8-909dacd7cf61/istio-proxy/1.log" Aug 03 17:04:21.695479 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:21.695397 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7596f8d454-5n7l8_584f7512-abe2-42d4-9503-778797f6f612/manager/0.log" Aug 03 17:04:21.752123 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:21.752073 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-sjrf7_d881ab55-1857-4390-b46a-b1c10a53b5cc/openshift-lws-operator/0.log" Aug 03 17:04:22.340999 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:22.340922 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-ndnrg_8758d0d7-c58c-4266-b081-d8d598272127/istio-proxy/0.log" Aug 03 17:04:22.697226 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:22.697154 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-4htkn_8db2abdc-9a5a-49a2-9df8-909dacd7cf61/istio-proxy/0.log" Aug 03 17:04:22.708524 ip-10-0-142-140 kubenswrapper[2582]: I0803 17:04:22.708501 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-4htkn_8db2abdc-9a5a-49a2-9df8-909dacd7cf61/istio-proxy/1.log"